2024/11/24-05:09:04.775446 139957257127488 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00: fork to child time = 0.006656 sec 2024/11/24-05:09:04.776241 139957257127488 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00 2024/11/24-05:09:04.781621 139957257127488 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00 2024/11/24-05:09:04.782522 139957257127488 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-05:09:04.782735 139957257127488 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-05:09:04.839804 139957257127488 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41989/att-00/MANIFEST-206347 2024/11/24-05:09:04.855289 139957257127488 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41989/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 293045, last_sequence is 4453442321, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 292847 2024/11/24-05:09:04.855295 139957257127488 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-05:09:04.855411 139957257127488 [version_set.cc:5721] Creating manifest 293045 2024/11/24-05:09:04.896193 139957257127488 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-05:09:04.920914 139957257127488 [compaction_job.cc:2437] [default] [JOB 41989] Compacting 1@2 + 23@3 files to L3, score 0.95, subcompactions 1 : 1 2024/11/24-05:09:04.920928 139957257127488 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 31480 Base level 2, inputs: [292995(27MB)], [293043(32MB) 292910(33MB) 292911(35MB) 292912(34MB) 292913(33MB) 292914(35MB) 292915(33MB) 292916(33MB) 292917(35MB) 292918(31MB) 292919(30MB) 292936(34MB) 292937(32MB) 292938(35MB) 292939(33MB) 292940(34MB) 292941(33MB) 292942(33MB) 292943(33MB) 292944(32MB) 292945(34MB) 292946(33MB) 293014(35MB)] 2024/11/24-05:09:04.928153 139957257127488 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-05:09:04.928178 139957257127488 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-05:09:04.928193 139957257127488 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-05:09:04.928198 139957257127488 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-05:09:04.928208 139957257127488 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-05:09:05.488028 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293046.sst) /dev/shm/ToplingTemp/Topling-276763-x93qQK 2024/11/24-05:09:06.603854 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293046.sst): first pass time = 1.12's, 120.291'MB/sec 2024/11/24-05:09:07.988197 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293046.sst): old prealloc_size = 147639500, real_size = 33044656 2024/11/24-05:09:07.988919 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293046.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927306 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 116.74 avg-zval = 33.48 2024/11/24-05:09:07.989820 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293046.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:09:09.052880 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293046.sst): second pass time = 1.38's, 78.594'MB/sec, value only(80.7% of KV) wait indexing time = 0.01's, remap KeyValue time = 1.06's, 34.888'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2582.590'MB/sec (index lex order gen) rebuild zvType time = 0.91's, 0.255'MB/sec write SST data time = 0.15's, 241.754'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36397.305'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.594'MB/sec zip pipeline throughput = 78.594'MB/sec entries = 927306 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.74 avg-zval = 33.48 usrkeys = 927306 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.74 avg-zval = 33.48 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927306 bytes = 3245584 } seq expand size = 6491142 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5123 value = 3.4866 dict = 1.00 all = 3.6134 } Zip/UnZip{ index = 0.0951 value = 0.2868 dict = 1.00 all = 0.2767 } ---------------------------- total value len = 2251.440318 GB avg = 0.200 KB (by entry num) total key len = 306.908927 GB avg = 0.027 KB total ukey len = 237.373273 GB avg = 0.015 KB total ukey num = 15.739588469 Billion total entry num = 11.280902417 Billion write speed all = 35.124386324 MB/sec (with seq num) write speed all = 33.885353693 MB/sec (without seq num) 2024/11/24-05:09:09.103937 139957257127488 [compaction_job.cc:1948] [default] [JOB 41989] Generated table #293046: 927306 keys, 37144726 bytes, temperature: kWarm 2024/11/24-05:09:09.104043 139957257127488 EVENT_LOG_v1 {"time_micros": 1732396149103982, "cf_name": "default", "job": 41989, "event": "table_file_creation", "file_number": 293046, "file_size": 37144726, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178719, "largest_seqno": 4453001483, "table_properties": {"data_size": 31048416, "index_size": 1764224, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25964568, "raw_average_key_size": 28, "raw_value_size": 108253304, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927306, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396145, "oldest_key_time": 0, "file_creation_time": 1732396145, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293046, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:09:09.144394 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293047.sst) /dev/shm/ToplingTemp/Topling-276763-eBSWiK 2024/11/24-05:09:10.044072 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293047.sst): first pass time = 0.90's, 149.191'MB/sec 2024/11/24-05:09:11.362342 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293047.sst): old prealloc_size = 147639500, real_size = 31471968 2024/11/24-05:09:11.362902 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293047.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920573 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0295 GB avg-val = 117.80 avg-zval = 32.02 2024/11/24-05:09:11.363680 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293047.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:09:12.182796 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293047.sst): second pass time = 1.32's, 82.305'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.334'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3285.789'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.13's, 281.063'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 34675.087'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.305'MB/sec zip pipeline throughput = 82.305'MB/sec entries = 920573 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.80 avg-zval = 32.02 usrkeys = 920573 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.80 avg-zval = 32.02 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 920573 bytes = 3222016 } seq expand size = 6444011 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.08 MB all = 0.0356 GB } UnZip/Zip{ index = 10.4436 value = 3.6787 dict = 1.00 all = 3.7752 } Zip/UnZip{ index = 0.0958 value = 0.2718 dict = 1.00 all = 0.2649 } ---------------------------- total value len = 2251.555203 GB avg = 0.200 KB (by entry num) total key len = 306.934703 GB avg = 0.027 KB total ukey len = 237.391685 GB avg = 0.015 KB total ukey num = 15.740509042 Billion total entry num = 11.281822990 Billion write speed all = 35.124808148 MB/sec (with seq num) write speed all = 33.885727651 MB/sec (without seq num) 2024/11/24-05:09:12.229708 139957257127488 [compaction_job.cc:1948] [default] [JOB 41989] Generated table #293047: 920573 keys, 35552186 bytes, temperature: kWarm 2024/11/24-05:09:12.229776 139957257127488 EVENT_LOG_v1 {"time_micros": 1732396152229741, "cf_name": "default", "job": 41989, "event": "table_file_creation", "file_number": 293047, "file_size": 35552186, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4308971951, "largest_seqno": 4452999160, "table_properties": {"data_size": 29478672, "index_size": 1762944, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25776044, "raw_average_key_size": 28, "raw_value_size": 108441806, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920573, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396149, "oldest_key_time": 0, "file_creation_time": 1732396149, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293047, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:09:12.271119 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293048.sst) /dev/shm/ToplingTemp/Topling-276763-tHOF7I 2024/11/24-05:09:13.275398 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293048.sst): first pass time = 1.00's, 133.651'MB/sec 2024/11/24-05:09:14.641425 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293048.sst): old prealloc_size = 147639500, real_size = 33071008 2024/11/24-05:09:14.641965 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293048.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926356 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 116.89 avg-zval = 33.55 2024/11/24-05:09:14.642747 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293048.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:09:15.495054 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293048.sst): second pass time = 1.37's, 79.309'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.542'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3279.782'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 291.976'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 28109.414'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.309'MB/sec zip pipeline throughput = 79.309'MB/sec entries = 926356 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.89 avg-zval = 33.55 usrkeys = 926356 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.89 avg-zval = 33.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926356 bytes = 3242256 } seq expand size = 6484492 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5142 value = 3.4842 dict = 1.00 all = 3.6112 } Zip/UnZip{ index = 0.0951 value = 0.2870 dict = 1.00 all = 0.2769 } ---------------------------- total value len = 2251.669968 GB avg = 0.200 KB (by entry num) total key len = 306.960641 GB avg = 0.027 KB total ukey len = 237.410212 GB avg = 0.015 KB total ukey num = 15.741435398 Billion total entry num = 11.282749346 Billion write speed all = 35.125142565 MB/sec (with seq num) write speed all = 33.886016673 MB/sec (without seq num) 2024/11/24-05:09:15.545079 139957257127488 [compaction_job.cc:1948] [default] [JOB 41989] Generated table #293048: 926356 keys, 37167552 bytes, temperature: kWarm 2024/11/24-05:09:15.545148 139957257127488 EVENT_LOG_v1 {"time_micros": 1732396155545113, "cf_name": "default", "job": 41989, "event": "table_file_creation", "file_number": 293048, "file_size": 37167552, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178730, "largest_seqno": 4453000489, "table_properties": {"data_size": 31077136, "index_size": 1762112, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25937968, "raw_average_key_size": 28, "raw_value_size": 108279836, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926356, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396152, "oldest_key_time": 0, "file_creation_time": 1732396152, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293048, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:09:15.602668 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293049.sst) /dev/shm/ToplingTemp/Topling-276763-6td5aH 2024/11/24-05:09:16.428859 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293049.sst): first pass time = 0.83's, 162.461'MB/sec 2024/11/24-05:09:17.760219 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293049.sst): old prealloc_size = 147639500, real_size = 32344464 2024/11/24-05:09:17.760772 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293049.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927264 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0303 GB avg-val = 116.75 avg-zval = 32.73 2024/11/24-05:09:17.761562 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293049.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:09:18.599409 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293049.sst): second pass time = 1.33's, 81.399'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.429'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3408.568'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 286.957'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 28867.702'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.399'MB/sec zip pipeline throughput = 81.399'MB/sec entries = 927264 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.75 avg-zval = 32.73 usrkeys = 927264 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.75 avg-zval = 32.73 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927264 bytes = 3245440 } seq expand size = 6490848 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5134 value = 3.5670 dict = 1.00 all = 3.6828 } Zip/UnZip{ index = 0.0951 value = 0.2803 dict = 1.00 all = 0.2715 } ---------------------------- total value len = 2251.784713 GB avg = 0.200 KB (by entry num) total key len = 306.986604 GB avg = 0.027 KB total ukey len = 237.428757 GB avg = 0.015 KB total ukey num = 15.742362662 Billion total entry num = 11.283676610 Billion write speed all = 35.125577285 MB/sec (with seq num) write speed all = 33.886402367 MB/sec (without seq num) 2024/11/24-05:09:18.649376 139957257127488 [compaction_job.cc:1948] [default] [JOB 41989] Generated table #293049: 927264 keys, 36444236 bytes, temperature: kWarm 2024/11/24-05:09:18.649448 139957257127488 EVENT_LOG_v1 {"time_micros": 1732396158649411, "cf_name": "default", "job": 41989, "event": "table_file_creation", "file_number": 293049, "file_size": 36444236, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680186, "largest_seqno": 4453001564, "table_properties": {"data_size": 30348480, "index_size": 1763968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25963392, "raw_average_key_size": 28, "raw_value_size": 108254476, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927264, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396155, "oldest_key_time": 0, "file_creation_time": 1732396155, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293049, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:09:18.687860 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293050.sst) /dev/shm/ToplingTemp/Topling-276763-CDU9mK 2024/11/24-05:09:19.708896 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293050.sst): first pass time = 1.02's, 131.459'MB/sec 2024/11/24-05:09:21.004959 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293050.sst): old prealloc_size = 147639500, real_size = 31462000 2024/11/24-05:09:21.005541 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293050.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.05 2024/11/24-05:09:21.006345 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293050.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:09:21.817566 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293050.sst): second pass time = 1.30's, 83.750'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.736'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3286.550'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 281.726'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 35372.302'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.750'MB/sec zip pipeline throughput = 83.750'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 32.05 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 32.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.3772 value = 3.6822 dict = 1.00 all = 3.7767 } Zip/UnZip{ index = 0.0964 value = 0.2716 dict = 1.00 all = 0.2648 } ---------------------------- total value len = 2251.899626 GB avg = 0.200 KB (by entry num) total key len = 307.012345 GB avg = 0.027 KB total ukey len = 237.447143 GB avg = 0.015 KB total ukey num = 15.743281962 Billion total entry num = 11.284595910 Billion write speed all = 35.125956330 MB/sec (with seq num) write speed all = 33.886735199 MB/sec (without seq num) 2024/11/24-05:09:21.864761 139957257127488 [compaction_job.cc:1948] [default] [JOB 41989] Generated table #293050: 919300 keys, 35538674 bytes, temperature: kWarm 2024/11/24-05:09:21.864821 139957257127488 EVENT_LOG_v1 {"time_micros": 1732396161864786, "cf_name": "default", "job": 41989, "event": "table_file_creation", "file_number": 293050, "file_size": 35538674, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4290888547, "largest_seqno": 4442797483, "table_properties": {"data_size": 29460192, "index_size": 1771776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396158, "oldest_key_time": 0, "file_creation_time": 1732396158, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293050, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:09:21.903616 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293051.sst) /dev/shm/ToplingTemp/Topling-276763-6GHTpH 2024/11/24-05:09:22.770336 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293051.sst): first pass time = 0.87's, 154.864'MB/sec 2024/11/24-05:09:24.097224 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293051.sst): old prealloc_size = 147639500, real_size = 33249344 2024/11/24-05:09:24.097816 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293051.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927399 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.73 avg-zval = 33.70 2024/11/24-05:09:24.098620 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293051.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:09:24.953599 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293051.sst): second pass time = 1.33's, 81.657'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.615'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3181.545'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 295.362'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33581.657'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.657'MB/sec zip pipeline throughput = 81.657'MB/sec entries = 927399 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.73 avg-zval = 33.70 usrkeys = 927399 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.73 avg-zval = 33.70 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927399 bytes = 3245904 } seq expand size = 6491793 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5134 value = 3.4637 dict = 1.00 all = 3.5936 } Zip/UnZip{ index = 0.0951 value = 0.2887 dict = 1.00 all = 0.2783 } ---------------------------- total value len = 2252.014368 GB avg = 0.200 KB (by entry num) total key len = 307.038312 GB avg = 0.027 KB total ukey len = 237.465691 GB avg = 0.015 KB total ukey num = 15.744209361 Billion total entry num = 11.285523309 Billion write speed all = 35.126375716 MB/sec (with seq num) write speed all = 33.887106091 MB/sec (without seq num) 2024/11/24-05:09:25.001691 139957257127488 [compaction_job.cc:1948] [default] [JOB 41989] Generated table #293051: 927399 keys, 37349418 bytes, temperature: kWarm 2024/11/24-05:09:25.001763 139957257127488 EVENT_LOG_v1 {"time_micros": 1732396165001726, "cf_name": "default", "job": 41989, "event": "table_file_creation", "file_number": 293051, "file_size": 37349418, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680199, "largest_seqno": 4453000837, "table_properties": {"data_size": 31253104, "index_size": 1764224, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25967172, "raw_average_key_size": 28, "raw_value_size": 108250682, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927399, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396161, "oldest_key_time": 0, "file_creation_time": 1732396161, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293051, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:09:25.042235 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293052.sst) /dev/shm/ToplingTemp/Topling-276763-QAWfVK 2024/11/24-05:09:26.043312 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293052.sst): first pass time = 1.00's, 134.080'MB/sec 2024/11/24-05:09:27.356868 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293052.sst): old prealloc_size = 147639500, real_size = 31675104 2024/11/24-05:09:27.357442 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293052.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927065 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0297 GB avg-val = 116.78 avg-zval = 32.01 2024/11/24-05:09:27.358225 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293052.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:09:28.185964 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293052.sst): second pass time = 1.31's, 82.463'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.150'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3185.961'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.13's, 279.917'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 28114.656'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.463'MB/sec zip pipeline throughput = 82.463'MB/sec entries = 927065 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.78 avg-zval = 32.01 usrkeys = 927065 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.78 avg-zval = 32.01 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927065 bytes = 3244736 } seq expand size = 6489455 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.08 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5153 value = 3.6476 dict = 1.00 all = 3.7518 } Zip/UnZip{ index = 0.0951 value = 0.2742 dict = 1.00 all = 0.2665 } ---------------------------- total value len = 2252.129117 GB avg = 0.200 KB (by entry num) total key len = 307.064270 GB avg = 0.027 KB total ukey len = 237.484233 GB avg = 0.015 KB total ukey num = 15.745136426 Billion total entry num = 11.286450374 Billion write speed all = 35.126748587 MB/sec (with seq num) write speed all = 33.887432148 MB/sec (without seq num) 2024/11/24-05:09:28.232817 139957257127488 [compaction_job.cc:1948] [default] [JOB 41989] Generated table #293052: 927065 keys, 35774294 bytes, temperature: kWarm 2024/11/24-05:09:28.232887 139957257127488 EVENT_LOG_v1 {"time_micros": 1732396168232852, "cf_name": "default", "job": 41989, "event": "table_file_creation", "file_number": 293052, "file_size": 35774294, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680212, "largest_seqno": 4453000661, "table_properties": {"data_size": 29679888, "index_size": 1763264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25957820, "raw_average_key_size": 28, "raw_value_size": 108259934, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927065, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396165, "oldest_key_time": 0, "file_creation_time": 1732396165, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293052, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:09:28.279835 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293053.sst) /dev/shm/ToplingTemp/Topling-276763-5CXRQK 2024/11/24-05:09:29.301193 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293053.sst): first pass time = 1.02's, 131.416'MB/sec 2024/11/24-05:09:30.600585 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293053.sst): old prealloc_size = 147639500, real_size = 31430992 2024/11/24-05:09:30.601162 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293053.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 32.03 2024/11/24-05:09:30.602485 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293053.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:09:31.416337 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293053.sst): second pass time = 1.30's, 83.529'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.529'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3164.012'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 277.896'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 33025.202'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.529'MB/sec zip pipeline throughput = 83.529'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.03 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4447 value = 3.6846 dict = 1.00 all = 3.7800 } Zip/UnZip{ index = 0.0957 value = 0.2714 dict = 1.00 all = 0.2646 } ---------------------------- total value len = 2252.244030 GB avg = 0.200 KB (by entry num) total key len = 307.090010 GB avg = 0.027 KB total ukey len = 237.502619 GB avg = 0.015 KB total ukey num = 15.746055726 Billion total entry num = 11.287369674 Billion write speed all = 35.127121643 MB/sec (with seq num) write speed all = 33.887759211 MB/sec (without seq num) 2024/11/24-05:09:31.464063 139957257127488 [compaction_job.cc:1948] [default] [JOB 41989] Generated table #293053: 919300 keys, 35507698 bytes, temperature: kWarm 2024/11/24-05:09:31.464135 139957257127488 EVENT_LOG_v1 {"time_micros": 1732396171464098, "cf_name": "default", "job": 41989, "event": "table_file_creation", "file_number": 293053, "file_size": 35507698, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4279908439, "largest_seqno": 4432211637, "table_properties": {"data_size": 29440640, "index_size": 1760320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396168, "oldest_key_time": 0, "file_creation_time": 1732396168, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293053, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:09:31.508026 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293054.sst) /dev/shm/ToplingTemp/Topling-276763-T7L1VH 2024/11/24-05:09:32.319231 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293054.sst): first pass time = 0.81's, 165.462'MB/sec 2024/11/24-05:09:33.663321 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293054.sst): old prealloc_size = 147639500, real_size = 33664368 2024/11/24-05:09:33.663840 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293054.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927479 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0317 GB avg-val = 116.71 avg-zval = 34.13 2024/11/24-05:09:33.664653 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293054.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:09:34.522131 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293054.sst): second pass time = 1.34's, 80.583'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.979'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3593.874'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 290.365'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34148.813'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.583'MB/sec zip pipeline throughput = 80.583'MB/sec entries = 927479 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.71 avg-zval = 34.13 usrkeys = 927479 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.71 avg-zval = 34.13 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927479 bytes = 3246192 } seq expand size = 6492353 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.4461 value = 3.4195 dict = 1.00 all = 3.5537 } Zip/UnZip{ index = 0.0957 value = 0.2924 dict = 1.00 all = 0.2814 } ---------------------------- total value len = 2252.358771 GB avg = 0.200 KB (by entry num) total key len = 307.115980 GB avg = 0.027 KB total ukey len = 237.521168 GB avg = 0.015 KB total ukey num = 15.746983205 Billion total entry num = 11.288297153 Billion write speed all = 35.127555509 MB/sec (with seq num) write speed all = 33.888144072 MB/sec (without seq num) 2024/11/24-05:09:34.572114 139957257127488 [compaction_job.cc:1948] [default] [JOB 41989] Generated table #293054: 927479 keys, 37768574 bytes, temperature: kWarm 2024/11/24-05:09:34.572184 139957257127488 EVENT_LOG_v1 {"time_micros": 1732396174572148, "cf_name": "default", "job": 41989, "event": "table_file_creation", "file_number": 293054, "file_size": 37768574, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4453001083, "table_properties": {"data_size": 31656544, "index_size": 1775744, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25969412, "raw_average_key_size": 28, "raw_value_size": 108248374, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927479, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396171, "oldest_key_time": 0, "file_creation_time": 1732396171, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293054, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:09:34.612767 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293055.sst) /dev/shm/ToplingTemp/Topling-276763-CMjXNI 2024/11/24-05:09:35.399413 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293055.sst): first pass time = 0.79's, 170.628'MB/sec 2024/11/24-05:09:36.674842 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293055.sst): old prealloc_size = 147639500, real_size = 28820832 2024/11/24-05:09:36.675442 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293055.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 939753 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1079 GB zip-val = 0.0268 GB avg-val = 114.82 avg-zval = 28.50 2024/11/24-05:09:36.676216 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293055.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:09:37.502261 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293055.sst): second pass time = 1.27's, 84.683'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 39.831'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3104.464'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.336'MB/sec write SST data time = 0.13's, 258.660'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 32442.818'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 84.683'MB/sec zip pipeline throughput = 84.683'MB/sec entries = 939753 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.82 avg-zval = 28.50 usrkeys = 939753 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.82 avg-zval = 28.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 939753 bytes = 3289152 } seq expand size = 6578271 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0188 GB value = 0.1079 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0268 GB dict = 1.08 MB all = 0.0330 GB } UnZip/Zip{ index = 10.4443 value = 4.0284 dict = 1.00 all = 4.0726 } Zip/UnZip{ index = 0.0957 value = 0.2482 dict = 1.00 all = 0.2455 } ---------------------------- total value len = 2252.473254 GB avg = 0.200 KB (by entry num) total key len = 307.142293 GB avg = 0.027 KB total ukey len = 237.539963 GB avg = 0.015 KB total ukey num = 15.747922958 Billion total entry num = 11.289236906 Billion write speed all = 35.128051103 MB/sec (with seq num) write speed all = 33.888587179 MB/sec (without seq num) 2024/11/24-05:09:37.545283 139957257127488 [compaction_job.cc:1948] [default] [JOB 41989] Generated table #293055: 939753 keys, 32955898 bytes, temperature: kWarm 2024/11/24-05:09:37.545352 139957257127488 EVENT_LOG_v1 {"time_micros": 1732396177545316, "cf_name": "default", "job": 41989, "event": "table_file_creation", "file_number": 293055, "file_size": 32955898, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680463, "largest_seqno": 4453001173, "table_properties": {"data_size": 26786128, "index_size": 1799552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26313084, "raw_average_key_size": 28, "raw_value_size": 107904702, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 939753, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396174, "oldest_key_time": 0, "file_creation_time": 1732396174, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293055, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:09:37.590463 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293056.sst) /dev/shm/ToplingTemp/Topling-276763-69X48G 2024/11/24-05:09:38.603968 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293056.sst): first pass time = 1.01's, 132.434'MB/sec 2024/11/24-05:09:39.856681 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293056.sst): old prealloc_size = 147639500, real_size = 27790624 2024/11/24-05:09:39.857282 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293056.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 945196 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1078 GB zip-val = 0.0257 GB avg-val = 114.00 avg-zval = 27.24 2024/11/24-05:09:39.858574 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293056.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:09:40.694119 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293056.sst): second pass time = 1.25's, 86.069'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 38.000'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3133.609'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.333'MB/sec write SST data time = 0.13's, 248.185'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.073'MB, throughput = 33110.953'MB/sec zip my value time = 1.25's, unzip length = 0.108'GB zip my value throughput = 86.069'MB/sec zip pipeline throughput = 86.069'MB/sec entries = 945196 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.00 avg-zval = 27.24 usrkeys = 945196 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.00 avg-zval = 27.24 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 945196 bytes = 3190048 } seq expand size = 6616372 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0257 GB dict = 1.07 MB all = 0.0318 GB } UnZip/Zip{ index = 10.4457 value = 4.1855 dict = 1.00 all = 4.2178 } Zip/UnZip{ index = 0.0957 value = 0.2389 dict = 1.00 all = 0.2371 } ---------------------------- total value len = 2252.587622 GB avg = 0.200 KB (by entry num) total key len = 307.168758 GB avg = 0.027 KB total ukey len = 237.558867 GB avg = 0.015 KB total ukey num = 15.748868154 Billion total entry num = 11.290182102 Billion write speed all = 35.128445103 MB/sec (with seq num) write speed all = 33.888931702 MB/sec (without seq num) 2024/11/24-05:09:40.735826 139957257127488 [compaction_job.cc:1948] [default] [JOB 41989] Generated table #293056: 945196 keys, 31821808 bytes, temperature: kWarm 2024/11/24-05:09:40.735895 139957257127488 EVENT_LOG_v1 {"time_micros": 1732396180735859, "cf_name": "default", "job": 41989, "event": "table_file_creation", "file_number": 293056, "file_size": 31821808, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4344398134, "largest_seqno": 4447155143, "table_properties": {"data_size": 25744400, "index_size": 1809728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26465488, "raw_average_key_size": 28, "raw_value_size": 107752344, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 945196, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396177, "oldest_key_time": 0, "file_creation_time": 1732396177, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293056, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:09:40.778640 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293057.sst) /dev/shm/ToplingTemp/Topling-276763-129BbJ 2024/11/24-05:09:41.751359 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293057.sst): first pass time = 0.97's, 137.987'MB/sec 2024/11/24-05:09:43.094599 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293057.sst): old prealloc_size = 147639500, real_size = 31938320 2024/11/24-05:09:43.095172 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293057.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920732 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0300 GB avg-val = 117.77 avg-zval = 32.54 2024/11/24-05:09:43.095923 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293057.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:09:43.913450 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293057.sst): second pass time = 1.34's, 80.798'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.991'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3190.425'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.13's, 283.347'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 32498.666'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.798'MB/sec zip pipeline throughput = 80.798'MB/sec entries = 920732 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.77 avg-zval = 32.54 usrkeys = 920732 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.77 avg-zval = 32.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 920732 bytes = 3222576 } seq expand size = 6445124 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.5114 value = 3.6199 dict = 1.00 all = 3.7261 } Zip/UnZip{ index = 0.0951 value = 0.2763 dict = 1.00 all = 0.2684 } ---------------------------- total value len = 2252.702505 GB avg = 0.200 KB (by entry num) total key len = 307.194539 GB avg = 0.027 KB total ukey len = 237.577282 GB avg = 0.015 KB total ukey num = 15.749788886 Billion total entry num = 11.291102834 Billion write speed all = 35.128823477 MB/sec (with seq num) write speed all = 33.889263756 MB/sec (without seq num) 2024/11/24-05:09:43.960740 139957257127488 [compaction_job.cc:1948] [default] [JOB 41989] Generated table #293057: 920732 keys, 36020716 bytes, temperature: kWarm 2024/11/24-05:09:43.960804 139957257127488 EVENT_LOG_v1 {"time_micros": 1732396183960767, "cf_name": "default", "job": 41989, "event": "table_file_creation", "file_number": 293057, "file_size": 36020716, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4452999953, "table_properties": {"data_size": 29956096, "index_size": 1751872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25780496, "raw_average_key_size": 28, "raw_value_size": 108437328, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920732, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396180, "oldest_key_time": 0, "file_creation_time": 1732396180, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293057, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:09:43.999203 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293058.sst) /dev/shm/ToplingTemp/Topling-276763-7PPCeK 2024/11/24-05:09:44.930545 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293058.sst): first pass time = 0.93's, 144.118'MB/sec 2024/11/24-05:09:46.218762 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293058.sst): old prealloc_size = 147639500, real_size = 29457584 2024/11/24-05:09:46.219379 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293058.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 936206 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1080 GB zip-val = 0.0274 GB avg-val = 115.36 avg-zval = 29.30 2024/11/24-05:09:46.220117 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293058.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:09:47.051748 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293058.sst): second pass time = 1.29's, 83.887'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 40.175'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3012.078'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.333'MB/sec write SST data time = 0.13's, 258.835'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 33801.450'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.887'MB/sec zip pipeline throughput = 83.887'MB/sec entries = 936206 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.36 avg-zval = 29.30 usrkeys = 936206 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.36 avg-zval = 29.30 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 936206 bytes = 3159712 } seq expand size = 6553442 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0274 GB dict = 1.08 MB all = 0.0335 GB } UnZip/Zip{ index = 10.4428 value = 3.9374 dict = 1.00 all = 4.0107 } Zip/UnZip{ index = 0.0958 value = 0.2540 dict = 1.00 all = 0.2493 } ---------------------------- total value len = 2252.817062 GB avg = 0.200 KB (by entry num) total key len = 307.220752 GB avg = 0.027 KB total ukey len = 237.596006 GB avg = 0.015 KB total ukey num = 15.750725092 Billion total entry num = 11.292039040 Billion write speed all = 35.129242368 MB/sec (with seq num) write speed all = 33.889633253 MB/sec (without seq num) 2024/11/24-05:09:47.095413 139957257127488 [compaction_job.cc:1948] [default] [JOB 41989] Generated table #293058: 936206 keys, 33464502 bytes, temperature: kWarm 2024/11/24-05:09:47.095480 139957257127488 EVENT_LOG_v1 {"time_micros": 1732396187095447, "cf_name": "default", "job": 41989, "event": "table_file_creation", "file_number": 293058, "file_size": 33464502, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4332858374, "largest_seqno": 4453001193, "table_properties": {"data_size": 27430304, "index_size": 1793024, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26213768, "raw_average_key_size": 28, "raw_value_size": 108003960, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 936206, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396183, "oldest_key_time": 0, "file_creation_time": 1732396183, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293058, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:09:47.133499 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293059.sst) /dev/shm/ToplingTemp/Topling-276763-AaVnbH 2024/11/24-05:09:47.669078 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293059.sst): first pass time = 0.54's, 250.620'MB/sec 2024/11/24-05:09:49.016008 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293059.sst): old prealloc_size = 147639500, real_size = 32662384 2024/11/24-05:09:49.016589 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293059.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932634 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0307 GB avg-val = 115.91 avg-zval = 32.87 2024/11/24-05:09:49.017370 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293059.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:09:49.877940 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293059.sst): second pass time = 1.35's, 80.307'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.668'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3182.624'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 286.357'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 28967.423'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.307'MB/sec zip pipeline throughput = 80.307'MB/sec entries = 932634 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.91 avg-zval = 32.87 usrkeys = 932634 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.91 avg-zval = 32.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932634 bytes = 3264240 } seq expand size = 6528438 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5136 value = 3.5265 dict = 1.00 all = 3.6495 } Zip/UnZip{ index = 0.0951 value = 0.2836 dict = 1.00 all = 0.2740 } ---------------------------- total value len = 2252.931695 GB avg = 0.199 KB (by entry num) total key len = 307.246866 GB avg = 0.027 KB total ukey len = 237.614659 GB avg = 0.015 KB total ukey num = 15.751657726 Billion total entry num = 11.292971674 Billion write speed all = 35.129811324 MB/sec (with seq num) write speed all = 33.890147903 MB/sec (without seq num) 2024/11/24-05:09:49.925373 139957257127488 [compaction_job.cc:1948] [default] [JOB 41989] Generated table #293059: 932634 keys, 36776984 bytes, temperature: kWarm 2024/11/24-05:09:49.925443 139957257127488 EVENT_LOG_v1 {"time_micros": 1732396189925407, "cf_name": "default", "job": 41989, "event": "table_file_creation", "file_number": 293059, "file_size": 36776984, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679865, "largest_seqno": 4452998732, "table_properties": {"data_size": 30654880, "index_size": 1774144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26113752, "raw_average_key_size": 28, "raw_value_size": 108104000, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932634, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396187, "oldest_key_time": 0, "file_creation_time": 1732396187, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293059, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:09:49.969423 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293060.sst) /dev/shm/ToplingTemp/Topling-276763-FFKXrH 2024/11/24-05:09:50.881630 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293060.sst): first pass time = 0.91's, 147.141'MB/sec 2024/11/24-05:09:52.194894 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293060.sst): old prealloc_size = 147639500, real_size = 31441584 2024/11/24-05:09:52.195506 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293060.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.04 2024/11/24-05:09:52.196256 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293060.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:09:53.011906 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293060.sst): second pass time = 1.31's, 82.649'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.334'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2986.781'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 276.986'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 33669.822'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.649'MB/sec zip pipeline throughput = 82.649'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.04 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.04 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4458 value = 3.6833 dict = 1.00 all = 3.7911 } Zip/UnZip{ index = 0.0957 value = 0.2715 dict = 1.00 all = 0.2638 } ---------------------------- total value len = 2253.046607 GB avg = 0.199 KB (by entry num) total key len = 307.272607 GB avg = 0.027 KB total ukey len = 237.633045 GB avg = 0.015 KB total ukey num = 15.752577026 Billion total entry num = 11.293890974 Billion write speed all = 35.130230601 MB/sec (with seq num) write speed all = 33.890519577 MB/sec (without seq num) 2024/11/24-05:09:53.057909 139957257127488 [compaction_job.cc:1948] [default] [JOB 41989] Generated table #293060: 919300 keys, 35403442 bytes, temperature: kWarm 2024/11/24-05:09:53.057990 139957257127488 EVENT_LOG_v1 {"time_micros": 1732396193057944, "cf_name": "default", "job": 41989, "event": "table_file_creation", "file_number": 293060, "file_size": 35403442, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4320799511, "largest_seqno": 4439740291, "table_properties": {"data_size": 29451424, "index_size": 1760128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396189, "oldest_key_time": 0, "file_creation_time": 1732396189, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293060, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:09:53.104143 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293061.sst) /dev/shm/ToplingTemp/Topling-276763-HbgyUI 2024/11/24-05:09:54.114423 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293061.sst): first pass time = 1.01's, 132.858'MB/sec 2024/11/24-05:09:55.447171 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293061.sst): old prealloc_size = 147639500, real_size = 31375968 2024/11/24-05:09:55.447762 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293061.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 936312 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0294 GB avg-val = 115.35 avg-zval = 31.36 2024/11/24-05:09:55.448558 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293061.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:09:56.325210 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293061.sst): second pass time = 1.33's, 81.081'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 40.434'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3130.624'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.13's, 264.455'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 32635.678'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.081'MB/sec zip pipeline throughput = 81.081'MB/sec entries = 936312 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.35 avg-zval = 31.36 usrkeys = 936312 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.35 avg-zval = 31.36 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 936312 bytes = 3277104 } seq expand size = 6554184 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.5107 value = 3.6785 dict = 1.00 all = 3.7806 } Zip/UnZip{ index = 0.0951 value = 0.2719 dict = 1.00 all = 0.2645 } ---------------------------- total value len = 2253.161162 GB avg = 0.199 KB (by entry num) total key len = 307.298823 GB avg = 0.027 KB total ukey len = 237.651771 GB avg = 0.015 KB total ukey num = 15.753513338 Billion total entry num = 11.294827286 Billion write speed all = 35.130565034 MB/sec (with seq num) write speed all = 33.890807594 MB/sec (without seq num) 2024/11/24-05:09:56.370884 139957257127488 [compaction_job.cc:1948] [default] [JOB 41989] Generated table #293061: 936312 keys, 35501858 bytes, temperature: kWarm 2024/11/24-05:09:56.370955 139957257127488 EVENT_LOG_v1 {"time_micros": 1732396196370920, "cf_name": "default", "job": 41989, "event": "table_file_creation", "file_number": 293061, "file_size": 35501858, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679880, "largest_seqno": 4453001211, "table_properties": {"data_size": 29360080, "index_size": 1781632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26216736, "raw_average_key_size": 28, "raw_value_size": 108001028, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 936312, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396193, "oldest_key_time": 0, "file_creation_time": 1732396193, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293061, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:09:56.425417 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293062.sst) /dev/shm/ToplingTemp/Topling-276763-Pl3rFJ 2024/11/24-05:09:57.398240 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293062.sst): first pass time = 0.97's, 137.975'MB/sec 2024/11/24-05:09:58.706311 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293062.sst): old prealloc_size = 147639500, real_size = 31356368 2024/11/24-05:09:58.706873 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293062.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.94 2024/11/24-05:09:58.707638 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293062.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:09:59.522602 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293062.sst): second pass time = 1.31's, 82.981'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.268'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3278.848'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 273.718'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 32601.484'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.981'MB/sec zip pipeline throughput = 82.981'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.94 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.94 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4443 value = 3.6940 dict = 1.00 all = 3.8002 } Zip/UnZip{ index = 0.0957 value = 0.2707 dict = 1.00 all = 0.2631 } ---------------------------- total value len = 2253.276075 GB avg = 0.199 KB (by entry num) total key len = 307.324564 GB avg = 0.027 KB total ukey len = 237.670157 GB avg = 0.015 KB total ukey num = 15.754432638 Billion total entry num = 11.295746586 Billion write speed all = 35.130953672 MB/sec (with seq num) write speed all = 33.891149716 MB/sec (without seq num) 2024/11/24-05:09:59.568785 139957257127488 [compaction_job.cc:1948] [default] [JOB 41989] Generated table #293062: 919300 keys, 35318194 bytes, temperature: kWarm 2024/11/24-05:09:59.568854 139957257127488 EVENT_LOG_v1 {"time_micros": 1732396199568819, "cf_name": "default", "job": 41989, "event": "table_file_creation", "file_number": 293062, "file_size": 35318194, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4290258803, "largest_seqno": 4401075506, "table_properties": {"data_size": 29365952, "index_size": 1760384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396196, "oldest_key_time": 0, "file_creation_time": 1732396196, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293062, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:09:59.609199 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293063.sst) /dev/shm/ToplingTemp/Topling-276763-JaLjMK 2024/11/24-05:10:00.527331 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293063.sst): first pass time = 0.92's, 146.192'MB/sec 2024/11/24-05:10:01.841197 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293063.sst): old prealloc_size = 147639500, real_size = 30373088 2024/11/24-05:10:01.841793 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293063.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932578 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0284 GB avg-val = 115.92 avg-zval = 30.42 2024/11/24-05:10:01.842598 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293063.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:10:02.673687 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293063.sst): second pass time = 1.31's, 82.367'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 41.428'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3109.243'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.332'MB/sec write SST data time = 0.13's, 267.167'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32238.373'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.367'MB/sec zip pipeline throughput = 82.367'MB/sec entries = 932578 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.92 avg-zval = 30.42 usrkeys = 932578 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.92 avg-zval = 30.42 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932578 bytes = 3264032 } seq expand size = 6528046 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0284 GB dict = 1.08 MB all = 0.0345 GB } UnZip/Zip{ index = 10.5137 value = 3.8111 dict = 1.00 all = 3.8918 } Zip/UnZip{ index = 0.0951 value = 0.2624 dict = 1.00 all = 0.2570 } ---------------------------- total value len = 2253.390709 GB avg = 0.199 KB (by entry num) total key len = 307.350676 GB avg = 0.027 KB total ukey len = 237.688808 GB avg = 0.015 KB total ukey num = 15.755365216 Billion total entry num = 11.296679164 Billion write speed all = 35.131365867 MB/sec (with seq num) write speed all = 33.891513155 MB/sec (without seq num) 2024/11/24-05:10:02.719126 139957257127488 [compaction_job.cc:1948] [default] [JOB 41989] Generated table #293063: 932578 keys, 34487634 bytes, temperature: kWarm 2024/11/24-05:10:02.719203 139957257127488 EVENT_LOG_v1 {"time_micros": 1732396202719163, "cf_name": "default", "job": 41989, "event": "table_file_creation", "file_number": 293063, "file_size": 34487634, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679891, "largest_seqno": 4453001224, "table_properties": {"data_size": 28365712, "index_size": 1774016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26112184, "raw_average_key_size": 28, "raw_value_size": 108105568, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932578, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396199, "oldest_key_time": 0, "file_creation_time": 1732396199, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293063, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:10:02.760412 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293064.sst) /dev/shm/ToplingTemp/Topling-276763-Ha2EkJ 2024/11/24-05:10:03.774712 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293064.sst): first pass time = 1.01's, 132.331'MB/sec 2024/11/24-05:10:05.076821 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293064.sst): old prealloc_size = 147639500, real_size = 31372368 2024/11/24-05:10:05.077415 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293064.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.96 2024/11/24-05:10:05.078175 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293064.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:10:05.918495 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293064.sst): second pass time = 1.30's, 83.357'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 41.982'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3056.457'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.324'MB/sec write SST data time = 0.13's, 267.966'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 30197.885'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.357'MB/sec zip pipeline throughput = 83.357'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.96 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.96 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4455 value = 3.6919 dict = 1.00 all = 3.7985 } Zip/UnZip{ index = 0.0957 value = 0.2709 dict = 1.00 all = 0.2633 } ---------------------------- total value len = 2253.505621 GB avg = 0.199 KB (by entry num) total key len = 307.376416 GB avg = 0.027 KB total ukey len = 237.707194 GB avg = 0.015 KB total ukey num = 15.756284516 Billion total entry num = 11.297598464 Billion write speed all = 35.131731582 MB/sec (with seq num) write speed all = 33.891833169 MB/sec (without seq num) 2024/11/24-05:10:05.965419 139957257127488 [compaction_job.cc:1948] [default] [JOB 41989] Generated table #293064: 919300 keys, 35334194 bytes, temperature: kWarm 2024/11/24-05:10:05.965488 139957257127488 EVENT_LOG_v1 {"time_micros": 1732396205965452, "cf_name": "default", "job": 41989, "event": "table_file_creation", "file_number": 293064, "file_size": 35334194, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4268543531, "largest_seqno": 4376643904, "table_properties": {"data_size": 29382144, "index_size": 1760192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396202, "oldest_key_time": 0, "file_creation_time": 1732396202, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293064, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:10:06.010953 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293065.sst) /dev/shm/ToplingTemp/Topling-276763-SRvFsI 2024/11/24-05:10:06.994892 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293065.sst): first pass time = 0.98's, 136.416'MB/sec 2024/11/24-05:10:08.292830 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293065.sst): old prealloc_size = 147639500, real_size = 30606784 2024/11/24-05:10:08.293433 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293065.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927609 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0286 GB avg-val = 116.69 avg-zval = 30.82 2024/11/24-05:10:08.294186 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293065.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:10:09.107633 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293065.sst): second pass time = 1.30's, 83.450'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 42.453'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3084.345'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.338'MB/sec write SST data time = 0.13's, 271.341'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33810.779'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.450'MB/sec zip pipeline throughput = 83.450'MB/sec entries = 927609 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.69 avg-zval = 30.82 usrkeys = 927609 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.69 avg-zval = 30.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927609 bytes = 3130688 } seq expand size = 6493263 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0286 GB dict = 1.08 MB all = 0.0346 GB } UnZip/Zip{ index = 10.3772 value = 3.7865 dict = 1.00 all = 3.8803 } Zip/UnZip{ index = 0.0964 value = 0.2641 dict = 1.00 all = 0.2577 } ---------------------------- total value len = 2253.620359 GB avg = 0.199 KB (by entry num) total key len = 307.402389 GB avg = 0.027 KB total ukey len = 237.725747 GB avg = 0.015 KB total ukey num = 15.757212125 Billion total entry num = 11.298526073 Billion write speed all = 35.132124894 MB/sec (with seq num) write speed all = 33.892178925 MB/sec (without seq num) 2024/11/24-05:10:09.152814 139957257127488 [compaction_job.cc:1948] [default] [JOB 41989] Generated table #293065: 927609 keys, 34589906 bytes, temperature: kWarm 2024/11/24-05:10:09.152885 139957257127488 EVENT_LOG_v1 {"time_micros": 1732396209152849, "cf_name": "default", "job": 41989, "event": "table_file_creation", "file_number": 293065, "file_size": 34589906, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4376643905, "largest_seqno": 4453001234, "table_properties": {"data_size": 28586928, "index_size": 1787776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25973052, "raw_average_key_size": 28, "raw_value_size": 108244722, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927609, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396205, "oldest_key_time": 0, "file_creation_time": 1732396205, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293065, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:10:09.189961 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293066.sst) /dev/shm/ToplingTemp/Topling-276763-GsISUJ 2024/11/24-05:10:10.197342 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293066.sst): first pass time = 1.01's, 133.241'MB/sec 2024/11/24-05:10:11.521677 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293066.sst): old prealloc_size = 147639500, real_size = 32529216 2024/11/24-05:10:11.522330 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293066.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923237 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0305 GB avg-val = 117.38 avg-zval = 33.07 2024/11/24-05:10:11.523121 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293066.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:10:12.356454 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293066.sst): second pass time = 1.32's, 81.874'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.867'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2805.487'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 287.429'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 31679.639'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.874'MB/sec zip pipeline throughput = 81.874'MB/sec entries = 923237 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.38 avg-zval = 33.07 usrkeys = 923237 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.38 avg-zval = 33.07 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923237 bytes = 3231344 } seq expand size = 6462659 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.4446 value = 3.5495 dict = 1.00 all = 3.6653 } Zip/UnZip{ index = 0.0957 value = 0.2817 dict = 1.00 all = 0.2728 } ---------------------------- total value len = 2253.735189 GB avg = 0.199 KB (by entry num) total key len = 307.428240 GB avg = 0.027 KB total ukey len = 237.744211 GB avg = 0.015 KB total ukey num = 15.758135362 Billion total entry num = 11.299449310 Billion write speed all = 35.132488987 MB/sec (with seq num) write speed all = 33.892496962 MB/sec (without seq num) 2024/11/24-05:10:12.404108 139957257127488 [compaction_job.cc:1948] [default] [JOB 41989] Generated table #293066: 923237 keys, 36618188 bytes, temperature: kWarm 2024/11/24-05:10:12.404178 139957257127488 EVENT_LOG_v1 {"time_micros": 1732396212404141, "cf_name": "default", "job": 41989, "event": "table_file_creation", "file_number": 293066, "file_size": 36618188, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679902, "largest_seqno": 4452999992, "table_properties": {"data_size": 30530352, "index_size": 1767872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25850636, "raw_average_key_size": 28, "raw_value_size": 108367126, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923237, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396209, "oldest_key_time": 0, "file_creation_time": 1732396209, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293066, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:10:12.450236 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293067.sst) /dev/shm/ToplingTemp/Topling-276763-qyGxoG 2024/11/24-05:10:13.441691 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293067.sst): first pass time = 0.99's, 135.381'MB/sec 2024/11/24-05:10:14.737095 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293067.sst): old prealloc_size = 147639500, real_size = 31003920 2024/11/24-05:10:14.737676 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293067.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922946 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0290 GB avg-val = 117.42 avg-zval = 31.43 2024/11/24-05:10:14.739240 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293067.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:10:15.558143 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293067.sst): second pass time = 1.29's, 83.713'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.600'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3164.570'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 271.337'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34735.438'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.713'MB/sec zip pipeline throughput = 83.713'MB/sec entries = 922946 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.42 avg-zval = 31.43 usrkeys = 922946 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.42 avg-zval = 31.43 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922946 bytes = 3114960 } seq expand size = 6460622 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0290 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.4451 value = 3.7363 dict = 1.00 all = 3.8375 } Zip/UnZip{ index = 0.0957 value = 0.2676 dict = 1.00 all = 0.2606 } ---------------------------- total value len = 2253.850025 GB avg = 0.199 KB (by entry num) total key len = 307.454083 GB avg = 0.027 KB total ukey len = 237.762670 GB avg = 0.015 KB total ukey num = 15.759058308 Billion total entry num = 11.300372256 Billion write speed all = 35.132875733 MB/sec (with seq num) write speed all = 33.892836886 MB/sec (without seq num) 2024/11/24-05:10:15.605360 139957257127488 [compaction_job.cc:1948] [default] [JOB 41989] Generated table #293067: 922946 keys, 34975024 bytes, temperature: kWarm 2024/11/24-05:10:15.605431 139957257127488 EVENT_LOG_v1 {"time_micros": 1732396215605396, "cf_name": "default", "job": 41989, "event": "table_file_creation", "file_number": 293067, "file_size": 34975024, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4356673553, "largest_seqno": 4453001247, "table_properties": {"data_size": 29005760, "index_size": 1767232, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25842488, "raw_average_key_size": 28, "raw_value_size": 108375296, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922946, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396212, "oldest_key_time": 0, "file_creation_time": 1732396212, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293067, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:10:15.645173 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293068.sst) /dev/shm/ToplingTemp/Topling-276763-gQIGQH 2024/11/24-05:10:16.472392 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293068.sst): first pass time = 0.83's, 162.260'MB/sec 2024/11/24-05:10:17.851219 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293068.sst): old prealloc_size = 147639500, real_size = 32937792 2024/11/24-05:10:17.851798 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293068.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928044 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.62 avg-zval = 33.33 2024/11/24-05:10:17.852641 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293068.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:10:18.718932 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293068.sst): second pass time = 1.38's, 78.539'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.687'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3246.225'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.13's, 281.786'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 31904.112'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.539'MB/sec zip pipeline throughput = 78.539'MB/sec entries = 928044 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.62 avg-zval = 33.33 usrkeys = 928044 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.62 avg-zval = 33.33 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928044 bytes = 3248176 } seq expand size = 6496308 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.4457 value = 3.4994 dict = 1.00 all = 3.6237 } Zip/UnZip{ index = 0.0957 value = 0.2858 dict = 1.00 all = 0.2760 } ---------------------------- total value len = 2253.964754 GB avg = 0.199 KB (by entry num) total key len = 307.480068 GB avg = 0.027 KB total ukey len = 237.781231 GB avg = 0.015 KB total ukey num = 15.759986352 Billion total entry num = 11.301300300 Billion write speed all = 35.133282644 MB/sec (with seq num) write speed all = 33.893195723 MB/sec (without seq num) 2024/11/24-05:10:18.765858 139957257127488 [compaction_job.cc:1948] [default] [JOB 41989] Generated table #293068: 928044 keys, 37039108 bytes, temperature: kWarm 2024/11/24-05:10:18.765930 139957257127488 EVENT_LOG_v1 {"time_micros": 1732396218765895, "cf_name": "default", "job": 41989, "event": "table_file_creation", "file_number": 293068, "file_size": 37039108, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679921, "largest_seqno": 4452999573, "table_properties": {"data_size": 30928688, "index_size": 1776896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25985232, "raw_average_key_size": 28, "raw_value_size": 108232544, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928044, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396215, "oldest_key_time": 0, "file_creation_time": 1732396215, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293068, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:10:18.803734 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293069.sst) /dev/shm/ToplingTemp/Topling-276763-pdh7HH 2024/11/24-05:10:18.839652 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293069.sst): first pass time = 0.04's, 220.944'MB/sec 2024/11/24-05:10:18.913079 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293069.sst): old prealloc_size = 147639500, real_size = 2506464 2024/11/24-05:10:18.913160 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293069.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 54296 prefix = 4 raw-key = 0.0009 GB zip-key = 0.0001 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.0064 GB zip-val = 0.0024 GB avg-val = 118.00 avg-zval = 43.97 2024/11/24-05:10:18.913236 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293069.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0001 GB, waited 0.000 sec, Key+Value = 0.008 GB 2024/11/24-05:10:18.961633 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00/293069.sst): second pass time = 0.07's, 87.969'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.05's, 55.942'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1667.077'MB/sec (index lex order gen) rebuild zvType time = 0.04's, 0.335'MB/sec write SST data time = 0.01's, 338.795'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.065'MB, throughput = 2878.432'MB/sec zip my value time = 0.07's, unzip length = 0.006'GB zip my value throughput = 87.969'MB/sec zip pipeline throughput = 87.969'MB/sec entries = 54296 avg-key = 28.00 avg-zkey = 1.94 avg-val = 118.00 avg-zval = 43.97 usrkeys = 54296 avg-key = 20.00 avg-zkey = 1.94 avg-val = 125.00 avg-zval = 43.97 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 54296 bytes = 156112 } seq expand size = 380072 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0011 GB value = 0.0064 GB dict = 0.06 MB all = 0.0079 GB } __ZipSize{ index = 0.0001 GB value = 0.0024 GB dict = 0.06 MB all = 0.0027 GB } UnZip/Zip{ index = 10.2896 value = 2.6839 dict = 1.00 all = 2.9196 } Zip/UnZip{ index = 0.0972 value = 0.3726 dict = 1.00 all = 0.3425 } ---------------------------- total value len = 2253.971541 GB avg = 0.199 KB (by entry num) total key len = 307.481588 GB avg = 0.027 KB total ukey len = 237.782317 GB avg = 0.015 KB total ukey num = 15.760040648 Billion total entry num = 11.301354596 Billion write speed all = 35.133279627 MB/sec (with seq num) write speed all = 33.893190877 MB/sec (without seq num) 2024/11/24-05:10:18.971851 139957257127488 [compaction_job.cc:1948] [default] [JOB 41989] Generated table #293069: 54296 keys, 2715170 bytes, temperature: kWarm 2024/11/24-05:10:18.971903 139957257127488 EVENT_LOG_v1 {"time_micros": 1732396218971876, "cf_name": "default", "job": 41989, "event": "table_file_creation", "file_number": 293069, "file_size": 2715170, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4336284440, "largest_seqno": 4342656009, "table_properties": {"data_size": 2387152, "index_size": 105536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 1520288, "raw_average_key_size": 28, "raw_value_size": 6406928, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 54296, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396218, "oldest_key_time": 0, "file_creation_time": 1732396218, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293069, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:10:18.983807 139957257127488 [compaction_job.cc:703] job-41989: subcompact[0], size: 0.821270 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41989/att-00 [293046,293047,293048,293049,293050,293051,293052,293053,293054,293055,293056,293057,293058,293059,293060,293061,293062,293063,293064,293065,293066,293067,293068,293069] 2024/11/24-05:10:19.120878 139957257127488 (Original Log Time 2024/11/24-05:09:04.920967) EVENT_LOG_v1 {"time_micros": 1732396144920939, "job": 41989, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [292995], "files_L3": [293043, 292910, 292911, 292912, 292913, 292914, 292915, 292916, 292917, 292918, 292919, 292936, 292937, 292938, 292939, 292940, 292941, 292942, 292943, 292944, 292945, 292946, 293014], "score": 0.951993, "input_data_size": 843829106, "oldest_snapshot_seqno": 4453442285} 2024/11/24-05:10:19.120882 139957257127488 (Original Log Time 2024/11/24-05:10:19.119312) [compaction_job.cc:2022] [default] [JOB 41989] Compacted 1@2 + 23@3 files to L3 => 821269984 bytes 2024/11/24-05:10:19.120883 139957257127488 (Original Log Time 2024/11/24-05:10:19.120808) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 24 0 0 0] max score 0.03, MB/sec: 11.4 rd, 11.1 wr, level 3, files in(1, 23) out(24 +0 blob) MB in(27.1, 777.7 +0.0 blob) out(783.2 +0.0 blob), read-write-amplify(58.7) write-amplify(28.9) OK, records in: 21480552, records dropped: 101067 output_compression: Snappy 2024/11/24-05:10:19.120887 139957257127488 (Original Log Time 2024/11/24-05:10:19.120852) EVENT_LOG_v1 {"time_micros": 1732396219120832, "cf": "default", "job": 41989, "event": "compaction_finished", "compaction_time_micros": 74062848, "compaction_time_cpu_micros": 32088145, "output_level": 3, "num_output_files": 24, "total_output_size": 821269984, "num_input_records": 21480552, "num_output_records": 21379485, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 560417253, "file_range_sync_nanos": 0, "file_fsync_nanos": 17337, "file_prepare_write_nanos": 59098, "lsm_state": [0, 0, 0, 24, 0, 0, 0]} 2024/11/24-05:10:19.247892 139957257127488 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732396144,"executesMs":74335,"compactionJobId":41989,"attempt":0,"compactionInputRawBytes":3109288052,"compactionInputZipBytes":843829106,"compactionOutputRawBytes":0,"compactionOutputZipBytes":821269984,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-e8d96b021c7fdc4cd367532ae4a27119-4ffe7437bfe063ba-00"}}, response = 2024/11/24-05:10:19.248011 139957257127488 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41989/att-00: olev 3, work 74.336 s, result 7.709 ms, install 1.590 ms, input{raw 2.896 GiB zip 804.738 MiB}