2024/11/23-15:32:34.982936 139957634799168 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00: fork to child time = 0.014713 sec 2024/11/23-15:32:34.984004 139957634799168 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00 2024/11/23-15:32:34.992806 139957634799168 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00 2024/11/23-15:32:34.994039 139957634799168 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-15:32:34.994407 139957634799168 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-15:32:35.050747 139957634799168 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-29415/att-00/MANIFEST-206347 2024/11/23-15:32:35.084919 139957634799168 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-29415/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 266782, last_sequence is 4071495726, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 266628 2024/11/23-15:32:35.084935 139957634799168 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-15:32:35.085179 139957634799168 [version_set.cc:5721] Creating manifest 266782 2024/11/23-15:32:35.125174 139957634799168 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-15:32:35.148359 139957634799168 [compaction_job.cc:2437] [default] [JOB 29415] Compacting 1@2 + 24@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-15:32:35.148388 139957634799168 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 21986 Base level 2, inputs: [266756(27MB)], [266703(38MB) 266704(37MB) 266705(38MB) 266706(34MB) 266707(33MB) 266708(32MB) 266709(34MB) 266710(36MB) 266711(35MB) 266712(35MB) 266713(35MB) 266714(36MB) 266715(36MB) 266716(38MB) 266717(35MB) 266718(34MB) 266719(37MB) 266720(33MB) 266721(37MB) 266722(35MB) 266723(34MB) 266724(36MB) 266725(37MB) 266778(38MB)] 2024/11/23-15:32:35.157409 139957634799168 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-15:32:35.157456 139957634799168 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-15:32:35.157486 139957634799168 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-15:32:35.157495 139957634799168 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-15:32:35.157514 139957634799168 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-15:32:35.801247 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266783.sst) /dev/shm/ToplingTemp/Topling-96235-1RsSv4 2024/11/23-15:32:37.057641 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266783.sst): first pass time = 1.26's, 106.832'MB/sec 2024/11/23-15:32:38.986235 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266783.sst): old prealloc_size = 147639500, real_size = 36744944 2024/11/23-15:32:38.986958 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266783.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926501 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0348 GB avg-val = 116.87 avg-zval = 37.51 2024/11/23-15:32:38.987791 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266783.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:32:40.096687 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266783.sst): second pass time = 1.93's, 56.193'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.11's, 36.468'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2435.178'MB/sec (index lex order gen) rebuild zvType time = 0.85's, 0.273'MB/sec write SST data time = 0.26's, 155.801'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35017.759'MB/sec zip my value time = 1.93's, unzip length = 0.108'GB zip my value throughput = 56.193'MB/sec zip pipeline throughput = 109.632'MB/sec entries = 926501 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.87 avg-zval = 37.51 usrkeys = 926501 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.87 avg-zval = 37.51 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926501 bytes = 2895328 } seq expand size = 6485507 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0348 GB dict = 1.08 MB all = 0.0405 GB } UnZip/Zip{ index = 10.5147 value = 3.1158 dict = 1.00 all = 3.3144 } Zip/UnZip{ index = 0.0951 value = 0.3209 dict = 1.00 all = 0.3017 } ---------------------------- total value len = 8.963381 GB avg = 0.168 KB (by entry num) total key len = 1.466763 GB avg = 0.028 KB total ukey len = 1.016408 GB avg = 0.015 KB total ukey num = 0.066476936 Billion total entry num = 0.053284665 Billion write speed all = 0.437361449 MB/sec (with seq num) write speed all = 0.419486598 MB/sec (without seq num) 2024/11/23-15:32:40.149590 139957634799168 [compaction_job.cc:1948] [default] [JOB 29415] Generated table #266783: 926501 keys, 40495058 bytes, temperature: kWarm 2024/11/23-15:32:40.149722 139957634799168 EVENT_LOG_v1 {"time_micros": 1732347160149648, "cf_name": "default", "job": 29415, "event": "table_file_creation", "file_number": 266783, "file_size": 40495058, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4051668818, "largest_seqno": 4071355467, "table_properties": {"data_size": 34750816, "index_size": 1762304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25942028, "raw_average_key_size": 28, "raw_value_size": 108275810, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926501, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347155, "oldest_key_time": 0, "file_creation_time": 1732347155, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266783, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:32:40.215326 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266784.sst) /dev/shm/ToplingTemp/Topling-96235-0P9Hv3 2024/11/23-15:32:41.583290 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266784.sst): first pass time = 1.37's, 98.119'MB/sec 2024/11/23-15:32:43.544310 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266784.sst): old prealloc_size = 147639500, real_size = 35746160 2024/11/23-15:32:43.545011 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266784.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922675 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0338 GB avg-val = 117.47 avg-zval = 36.59 2024/11/23-15:32:43.545836 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266784.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:32:44.750686 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266784.sst): second pass time = 1.96's, 55.298'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.21's, 32.829'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2588.002'MB/sec (index lex order gen) rebuild zvType time = 1.02's, 0.226'MB/sec write SST data time = 0.18's, 215.615'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 36074.276'MB/sec zip my value time = 1.96's, unzip length = 0.108'GB zip my value throughput = 55.298'MB/sec zip pipeline throughput = 92.719'MB/sec entries = 922675 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.47 avg-zval = 36.59 usrkeys = 922675 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.47 avg-zval = 36.59 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922675 bytes = 2998704 } seq expand size = 6458725 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0338 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.5129 value = 3.2104 dict = 1.00 all = 3.3891 } Zip/UnZip{ index = 0.0951 value = 0.3115 dict = 1.00 all = 0.2951 } ---------------------------- total value len = 9.078223 GB avg = 0.167 KB (by entry num) total key len = 1.492598 GB avg = 0.028 KB total ukey len = 1.034862 GB avg = 0.015 KB total ukey num = 0.067399611 Billion total entry num = 0.054207340 Billion write speed all = 0.443173872 MB/sec (with seq num) write speed all = 0.424993049 MB/sec (without seq num) 2024/11/23-15:32:44.802445 139957634799168 [compaction_job.cc:1948] [default] [JOB 29415] Generated table #266784: 922675 keys, 39603074 bytes, temperature: kWarm 2024/11/23-15:32:44.802508 139957634799168 EVENT_LOG_v1 {"time_micros": 1732347164802473, "cf_name": "default", "job": 29415, "event": "table_file_creation", "file_number": 266784, "file_size": 39603074, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4012777911, "largest_seqno": 4071355507, "table_properties": {"data_size": 33759968, "index_size": 1755328, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25834900, "raw_average_key_size": 28, "raw_value_size": 108382898, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922675, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347160, "oldest_key_time": 0, "file_creation_time": 1732347160, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266784, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:32:44.843665 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266785.sst) /dev/shm/ToplingTemp/Topling-96235-3NDIh3 2024/11/23-15:32:46.069667 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266785.sst): first pass time = 1.23's, 109.479'MB/sec 2024/11/23-15:32:47.539816 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266785.sst): old prealloc_size = 147639500, real_size = 36556064 2024/11/23-15:32:47.540501 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266785.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928163 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0345 GB avg-val = 116.61 avg-zval = 37.22 2024/11/23-15:32:47.541418 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266785.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:32:48.491347 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266785.sst): second pass time = 1.47's, 73.663'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 42.977'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2695.935'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.291'MB/sec write SST data time = 0.15's, 266.789'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 37829.595'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.663'MB/sec zip pipeline throughput = 73.663'MB/sec entries = 928163 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.61 avg-zval = 37.22 usrkeys = 928163 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.61 avg-zval = 37.22 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928163 bytes = 3480624 } seq expand size = 6497141 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0409 GB } UnZip/Zip{ index = 10.4455 value = 3.1328 dict = 1.00 all = 3.2821 } Zip/UnZip{ index = 0.0957 value = 0.3192 dict = 1.00 all = 0.3047 } ---------------------------- total value len = 9.192949 GB avg = 0.167 KB (by entry num) total key len = 1.518587 GB avg = 0.028 KB total ukey len = 1.053425 GB avg = 0.015 KB total ukey num = 0.068327774 Billion total entry num = 0.055135503 Billion write speed all = 0.449002829 MB/sec (with seq num) write speed all = 0.430513606 MB/sec (without seq num) 2024/11/23-15:32:48.543918 139957634799168 [compaction_job.cc:1948] [default] [JOB 29415] Generated table #266785: 928163 keys, 40893578 bytes, temperature: kWarm 2024/11/23-15:32:48.544009 139957634799168 EVENT_LOG_v1 {"time_micros": 1732347168543954, "cf_name": "default", "job": 29415, "event": "table_file_creation", "file_number": 266785, "file_size": 40893578, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 4071355521, "table_properties": {"data_size": 34546704, "index_size": 1777152, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25988564, "raw_average_key_size": 28, "raw_value_size": 108229274, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928163, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347164, "oldest_key_time": 0, "file_creation_time": 1732347164, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266785, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:32:48.583664 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266786.sst) /dev/shm/ToplingTemp/Topling-96235-Vc4ln2 2024/11/23-15:32:49.810370 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266786.sst): first pass time = 1.23's, 109.417'MB/sec 2024/11/23-15:32:51.274682 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266786.sst): old prealloc_size = 147639500, real_size = 32550528 2024/11/23-15:32:51.275422 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266786.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933471 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0305 GB avg-val = 115.78 avg-zval = 32.72 2024/11/23-15:32:51.276311 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266786.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:32:52.165138 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266786.sst): second pass time = 1.46's, 73.855'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 41.311'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2481.935'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.317'MB/sec write SST data time = 0.15's, 240.926'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 39933.089'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 73.855'MB/sec zip pipeline throughput = 73.855'MB/sec entries = 933471 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.78 avg-zval = 32.72 usrkeys = 933471 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.78 avg-zval = 32.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933471 bytes = 3383840 } seq expand size = 6534297 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5143 value = 3.5388 dict = 1.00 all = 3.6487 } Zip/UnZip{ index = 0.0951 value = 0.2826 dict = 1.00 all = 0.2741 } ---------------------------- total value len = 9.307564 GB avg = 0.166 KB (by entry num) total key len = 1.544724 GB avg = 0.028 KB total ukey len = 1.072094 GB avg = 0.015 KB total ukey num = 0.069261245 Billion total entry num = 0.056068974 Billion write speed all = 0.454832790 MB/sec (with seq num) write speed all = 0.436033430 MB/sec (without seq num) 2024/11/23-15:32:52.210712 139957634799168 [compaction_job.cc:1948] [default] [JOB 29415] Generated table #266786: 933471 keys, 36784730 bytes, temperature: kWarm 2024/11/23-15:32:52.210806 139957634799168 EVENT_LOG_v1 {"time_micros": 1732347172210747, "cf_name": "default", "job": 29415, "event": "table_file_creation", "file_number": 266786, "file_size": 36784730, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3563652023, "largest_seqno": 4012776165, "table_properties": {"data_size": 30541360, "index_size": 1775616, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26137188, "raw_average_key_size": 28, "raw_value_size": 108080642, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933471, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347168, "oldest_key_time": 0, "file_creation_time": 1732347168, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266786, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:32:52.247131 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266787.sst) /dev/shm/ToplingTemp/Topling-96235-kId4c2 2024/11/23-15:32:53.395331 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266787.sst): first pass time = 1.15's, 116.898'MB/sec 2024/11/23-15:32:54.902532 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266787.sst): old prealloc_size = 147639500, real_size = 31854768 2024/11/23-15:32:54.903229 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266787.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934763 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0298 GB avg-val = 115.58 avg-zval = 31.93 2024/11/23-15:32:54.904057 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266787.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:32:55.780674 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266787.sst): second pass time = 1.51's, 71.724'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 40.970'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2622.770'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.15's, 243.092'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33248.265'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.724'MB/sec zip pipeline throughput = 71.724'MB/sec entries = 934763 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.58 avg-zval = 31.93 usrkeys = 934763 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.58 avg-zval = 31.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934763 bytes = 3271680 } seq expand size = 6543341 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0298 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.5122 value = 3.6205 dict = 1.00 all = 3.7307 } Zip/UnZip{ index = 0.0951 value = 0.2762 dict = 1.00 all = 0.2680 } ---------------------------- total value len = 9.422152 GB avg = 0.165 KB (by entry num) total key len = 1.570897 GB avg = 0.028 KB total ukey len = 1.090789 GB avg = 0.016 KB total ukey num = 0.070196008 Billion total entry num = 0.057003737 Billion write speed all = 0.460662458 MB/sec (with seq num) write speed all = 0.441552577 MB/sec (without seq num) 2024/11/23-15:32:55.825845 139957634799168 [compaction_job.cc:1948] [default] [JOB 29415] Generated table #266787: 934763 keys, 35976646 bytes, temperature: kWarm 2024/11/23-15:32:55.825910 139957634799168 EVENT_LOG_v1 {"time_micros": 1732347175825877, "cf_name": "default", "job": 29415, "event": "table_file_creation", "file_number": 266787, "file_size": 35976646, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3759280178, "largest_seqno": 4012776176, "table_properties": {"data_size": 29842464, "index_size": 1778432, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26173364, "raw_average_key_size": 28, "raw_value_size": 108044426, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934763, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347172, "oldest_key_time": 0, "file_creation_time": 1732347172, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266787, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:32:55.862087 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266788.sst) /dev/shm/ToplingTemp/Topling-96235-0yUkr2 2024/11/23-15:32:56.995764 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266788.sst): first pass time = 1.13's, 118.396'MB/sec 2024/11/23-15:32:58.465576 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266788.sst): old prealloc_size = 147639500, real_size = 31329120 2024/11/23-15:32:58.466300 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266788.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931885 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0293 GB avg-val = 116.03 avg-zval = 31.47 2024/11/23-15:32:58.467153 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266788.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:32:59.333945 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266788.sst): second pass time = 1.47's, 73.608'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 40.815'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2572.767'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.15's, 229.738'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 27926.736'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.608'MB/sec zip pipeline throughput = 73.608'MB/sec entries = 931885 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.03 avg-zval = 31.47 usrkeys = 931885 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.03 avg-zval = 31.47 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931885 bytes = 3261616 } seq expand size = 6523195 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.08 MB all = 0.0354 GB } UnZip/Zip{ index = 10.5143 value = 3.6873 dict = 1.00 all = 3.7870 } Zip/UnZip{ index = 0.0951 value = 0.2712 dict = 1.00 all = 0.2641 } ---------------------------- total value len = 9.536800 GB avg = 0.165 KB (by entry num) total key len = 1.596990 GB avg = 0.028 KB total ukey len = 1.109427 GB avg = 0.016 KB total ukey num = 0.071127893 Billion total entry num = 0.057935622 Billion write speed all = 0.466490735 MB/sec (with seq num) write speed all = 0.447071342 MB/sec (without seq num) 2024/11/23-15:32:59.378994 139957634799168 [compaction_job.cc:1948] [default] [JOB 29415] Generated table #266788: 931885 keys, 35441508 bytes, temperature: kWarm 2024/11/23-15:32:59.379062 139957634799168 EVENT_LOG_v1 {"time_micros": 1732347179379028, "cf_name": "default", "job": 29415, "event": "table_file_creation", "file_number": 266788, "file_size": 35441508, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3801926538, "largest_seqno": 4012776192, "table_properties": {"data_size": 29323344, "index_size": 1772608, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26092780, "raw_average_key_size": 28, "raw_value_size": 108125074, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931885, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347175, "oldest_key_time": 0, "file_creation_time": 1732347175, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266788, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:32:59.415602 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266789.sst) /dev/shm/ToplingTemp/Topling-96235-FOFDA4 2024/11/23-15:33:00.589134 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266789.sst): first pass time = 1.17's, 114.376'MB/sec 2024/11/23-15:33:02.293999 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266789.sst): old prealloc_size = 147639500, real_size = 35108496 2024/11/23-15:33:02.294679 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266789.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1083224 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1039 GB zip-val = 0.0328 GB avg-val = 95.91 avg-zval = 30.23 2024/11/23-15:33:02.294905 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266789.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:33:03.024526 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266789.sst): second pass time = 1.70's, 60.966'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.73's, 49.712'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3182.552'MB/sec (index lex order gen) rebuild zvType time = 0.58's, 0.464'MB/sec write SST data time = 0.15's, 249.245'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.033'MB, throughput = 31634.234'MB/sec zip my value time = 1.70's, unzip length = 0.104'GB zip my value throughput = 60.966'MB/sec zip pipeline throughput = 70.938'MB/sec entries = 1083224 avg-key = 28.00 avg-zkey = 1.93 avg-val = 95.91 avg-zval = 30.23 usrkeys = 1083224 avg-key = 20.00 avg-zkey = 1.93 avg-val = 96.48 avg-zval = 30.23 TagRS{ kinds = RS_Key0_TagN bytes = 169360 } TagArray{ size = 88091 bytes = 264288 } seq expand size = 616637 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0217 GB value = 0.1039 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0328 GB dict = 1.03 MB all = 0.0363 GB } UnZip/Zip{ index = 10.3795 value = 3.1721 dict = 1.00 all = 3.6959 } Zip/UnZip{ index = 0.0963 value = 0.3152 dict = 1.00 all = 0.2706 } ---------------------------- total value len = 9.641304 GB avg = 0.163 KB (by entry num) total key len = 1.627320 GB avg = 0.028 KB total ukey len = 1.131092 GB avg = 0.016 KB total ukey num = 0.072211117 Billion total entry num = 0.059018846 Billion write speed all = 0.472067121 MB/sec (with seq num) write speed all = 0.452287701 MB/sec (without seq num) 2024/11/23-15:33:03.072457 139957634799168 [compaction_job.cc:1948] [default] [JOB 29415] Generated table #266789: 1083224 keys, 36315524 bytes, temperature: kWarm 2024/11/23-15:33:03.072523 139957634799168 EVENT_LOG_v1 {"time_micros": 1732347183072490, "cf_name": "default", "job": 29415, "event": "table_file_creation", "file_number": 266789, "file_size": 36315524, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012775259, "table_properties": {"data_size": 32750272, "index_size": 2087232, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30330272, "raw_average_key_size": 28, "raw_value_size": 103887561, "raw_average_value_size": 95, "num_data_blocks": 1, "num_entries": 1083224, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347179, "oldest_key_time": 0, "file_creation_time": 1732347179, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266789, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:33:03.108220 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266790.sst) /dev/shm/ToplingTemp/Topling-96235-R28CH0 2024/11/23-15:33:04.230541 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266790.sst): first pass time = 1.12's, 119.595'MB/sec 2024/11/23-15:33:05.799784 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266790.sst): old prealloc_size = 147639500, real_size = 35187616 2024/11/23-15:33:05.800507 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266790.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922611 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0332 GB avg-val = 117.48 avg-zval = 35.99 2024/11/23-15:33:05.801125 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266790.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:33:06.596819 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266790.sst): second pass time = 1.57's, 69.105'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.80's, 47.902'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2509.861'MB/sec (index lex order gen) rebuild zvType time = 0.67's, 0.346'MB/sec write SST data time = 0.13's, 293.439'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 38632.563'MB/sec zip my value time = 1.57's, unzip length = 0.108'GB zip my value throughput = 69.105'MB/sec zip pipeline throughput = 69.105'MB/sec entries = 922611 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.48 avg-zval = 35.99 usrkeys = 922611 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.49 avg-zval = 35.99 TagRS{ kinds = RS_Key0_TagN bytes = 144240 } TagArray{ size = 529341 bytes = 1985040 } seq expand size = 3705387 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.5133 value = 3.2644 dict = 1.00 all = 3.5155 } Zip/UnZip{ index = 0.0951 value = 0.3063 dict = 1.00 all = 0.2845 } ---------------------------- total value len = 9.753394 GB avg = 0.163 KB (by entry num) total key len = 1.653153 GB avg = 0.028 KB total ukey len = 1.149544 GB avg = 0.016 KB total ukey num = 0.073133728 Billion total entry num = 0.059941457 Billion write speed all = 0.477773525 MB/sec (with seq num) write speed all = 0.457687909 MB/sec (without seq num) 2024/11/23-15:33:06.645212 139957634799168 [compaction_job.cc:1948] [default] [JOB 29415] Generated table #266790: 922611 keys, 38178708 bytes, temperature: kWarm 2024/11/23-15:33:06.645274 139957634799168 EVENT_LOG_v1 {"time_micros": 1732347186645243, "cf_name": "default", "job": 29415, "event": "table_file_creation", "file_number": 266790, "file_size": 38178708, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3989181692, "table_properties": {"data_size": 33201616, "index_size": 1755136, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25833108, "raw_average_key_size": 28, "raw_value_size": 108384722, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922611, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347183, "oldest_key_time": 0, "file_creation_time": 1732347183, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266790, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:33:06.685811 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266791.sst) /dev/shm/ToplingTemp/Topling-96235-jcWBj2 2024/11/23-15:33:07.783878 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266791.sst): first pass time = 1.10's, 122.236'MB/sec 2024/11/23-15:33:09.369180 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266791.sst): old prealloc_size = 147639500, real_size = 32788080 2024/11/23-15:33:09.370065 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266791.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931429 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 116.10 avg-zval = 33.04 2024/11/23-15:33:09.371075 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266791.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:33:10.248596 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266791.sst): second pass time = 1.58's, 68.287'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 42.092'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2079.018'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.316'MB/sec write SST data time = 0.14's, 261.227'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 37144.117'MB/sec zip my value time = 1.58's, unzip length = 0.108'GB zip my value throughput = 68.287'MB/sec zip pipeline throughput = 69.442'MB/sec entries = 931429 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.10 avg-zval = 33.04 usrkeys = 931429 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.10 avg-zval = 33.04 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931429 bytes = 3376448 } seq expand size = 6520003 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.4462 value = 3.5142 dict = 1.00 all = 3.6260 } Zip/UnZip{ index = 0.0957 value = 0.2846 dict = 1.00 all = 0.2758 } ---------------------------- total value len = 9.868052 GB avg = 0.162 KB (by entry num) total key len = 1.679233 GB avg = 0.028 KB total ukey len = 1.168172 GB avg = 0.016 KB total ukey num = 0.074065157 Billion total entry num = 0.060872886 Billion write speed all = 0.483594483 MB/sec (with seq num) write speed all = 0.463199878 MB/sec (without seq num) 2024/11/23-15:33:10.293855 139957634799168 [compaction_job.cc:1948] [default] [JOB 29415] Generated table #266791: 931429 keys, 37015660 bytes, temperature: kWarm 2024/11/23-15:33:10.293928 139957634799168 EVENT_LOG_v1 {"time_micros": 1732347190293890, "cf_name": "default", "job": 29415, "event": "table_file_creation", "file_number": 266791, "file_size": 37015660, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3487497056, "largest_seqno": 4012775885, "table_properties": {"data_size": 30771744, "index_size": 1783296, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26080012, "raw_average_key_size": 28, "raw_value_size": 108137762, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347186, "oldest_key_time": 0, "file_creation_time": 1732347186, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266791, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:33:10.332919 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266792.sst) /dev/shm/ToplingTemp/Topling-96235-bA5ka3 2024/11/23-15:33:11.329077 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266792.sst): first pass time = 1.00's, 134.741'MB/sec 2024/11/23-15:33:12.839352 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266792.sst): old prealloc_size = 147639500, real_size = 32763456 2024/11/23-15:33:12.840013 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266792.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931309 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 116.12 avg-zval = 33.03 2024/11/23-15:33:12.840816 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266792.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:33:13.720808 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266792.sst): second pass time = 1.51's, 71.637'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 41.835'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2763.876'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.15's, 247.763'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36133.615'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.637'MB/sec zip pipeline throughput = 71.637'MB/sec entries = 931309 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.12 avg-zval = 33.03 usrkeys = 931309 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.12 avg-zval = 33.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931309 bytes = 3259600 } seq expand size = 6519163 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5116 value = 3.5158 dict = 1.00 all = 3.6399 } Zip/UnZip{ index = 0.0951 value = 0.2844 dict = 1.00 all = 0.2747 } ---------------------------- total value len = 9.982713 GB avg = 0.162 KB (by entry num) total key len = 1.705310 GB avg = 0.028 KB total ukey len = 1.186799 GB avg = 0.016 KB total ukey num = 0.074996466 Billion total entry num = 0.061804195 Billion write speed all = 0.489417308 MB/sec (with seq num) write speed all = 0.468713691 MB/sec (without seq num) 2024/11/23-15:33:13.766022 139957634799168 [compaction_job.cc:1948] [default] [JOB 29415] Generated table #266792: 931309 keys, 36874528 bytes, temperature: kWarm 2024/11/23-15:33:13.766091 139957634799168 EVENT_LOG_v1 {"time_micros": 1732347193766055, "cf_name": "default", "job": 29415, "event": "table_file_creation", "file_number": 266792, "file_size": 36874528, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3807172320, "largest_seqno": 4012775902, "table_properties": {"data_size": 30758448, "index_size": 1771968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26076652, "raw_average_key_size": 28, "raw_value_size": 108141150, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931309, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347190, "oldest_key_time": 0, "file_creation_time": 1732347190, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266792, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:33:13.803871 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266793.sst) /dev/shm/ToplingTemp/Topling-96235-TLqwa4 2024/11/23-15:33:14.865483 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266793.sst): first pass time = 1.06's, 126.433'MB/sec 2024/11/23-15:33:16.168723 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266793.sst): old prealloc_size = 147639500, real_size = 32862992 2024/11/23-15:33:16.169304 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266793.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931462 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0309 GB avg-val = 116.09 avg-zval = 33.13 2024/11/23-15:33:16.170154 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266793.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:33:17.026446 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266793.sst): second pass time = 1.30's, 83.051'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.380'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3145.289'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.13's, 286.785'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33043.462'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.051'MB/sec zip pipeline throughput = 83.051'MB/sec entries = 931462 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.09 avg-zval = 33.13 usrkeys = 931462 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.09 avg-zval = 33.13 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931462 bytes = 3492992 } seq expand size = 6520234 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5133 value = 3.5043 dict = 1.00 all = 3.6073 } Zip/UnZip{ index = 0.0951 value = 0.2854 dict = 1.00 all = 0.2772 } ---------------------------- total value len = 10.097370 GB avg = 0.161 KB (by entry num) total key len = 1.731391 GB avg = 0.028 KB total ukey len = 1.205428 GB avg = 0.016 KB total ukey num = 0.075927928 Billion total entry num = 0.062735657 Billion write speed all = 0.495241938 MB/sec (with seq num) write speed all = 0.474229202 MB/sec (without seq num) 2024/11/23-15:33:17.072125 139957634799168 [compaction_job.cc:1948] [default] [JOB 29415] Generated table #266793: 931462 keys, 37207052 bytes, temperature: kWarm 2024/11/23-15:33:17.072193 139957634799168 EVENT_LOG_v1 {"time_micros": 1732347197072158, "cf_name": "default", "job": 29415, "event": "table_file_creation", "file_number": 266793, "file_size": 37207052, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 4012775918, "table_properties": {"data_size": 30857984, "index_size": 1771968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26080936, "raw_average_key_size": 28, "raw_value_size": 108136860, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931462, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347193, "oldest_key_time": 0, "file_creation_time": 1732347193, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266793, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:33:17.109787 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266794.sst) /dev/shm/ToplingTemp/Topling-96235-dRSsz2 2024/11/23-15:33:18.156202 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266794.sst): first pass time = 1.05's, 128.269'MB/sec 2024/11/23-15:33:19.490534 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266794.sst): old prealloc_size = 147639500, real_size = 34600448 2024/11/23-15:33:19.491094 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266794.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925405 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0326 GB avg-val = 117.04 avg-zval = 35.24 2024/11/23-15:33:19.491861 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266794.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:33:20.335323 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266794.sst): second pass time = 1.33's, 81.244'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 45.942'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3285.478'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 303.580'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33527.175'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.244'MB/sec zip pipeline throughput = 81.244'MB/sec entries = 925405 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.04 avg-zval = 35.24 usrkeys = 925405 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.04 avg-zval = 35.24 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925405 bytes = 3354608 } seq expand size = 6477835 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0326 GB dict = 1.08 MB all = 0.0388 GB } UnZip/Zip{ index = 10.5133 value = 3.3214 dict = 1.00 all = 3.4583 } Zip/UnZip{ index = 0.0951 value = 0.3011 dict = 1.00 all = 0.2892 } ---------------------------- total value len = 10.212154 GB avg = 0.160 KB (by entry num) total key len = 1.757302 GB avg = 0.028 KB total ukey len = 1.223936 GB avg = 0.016 KB total ukey num = 0.076853333 Billion total entry num = 0.063661062 Billion write speed all = 0.501063112 MB/sec (with seq num) write speed all = 0.479743373 MB/sec (without seq num) 2024/11/23-15:33:20.383526 139957634799168 [compaction_job.cc:1948] [default] [JOB 29415] Generated table #266794: 925405 keys, 38810334 bytes, temperature: kWarm 2024/11/23-15:33:20.383596 139957634799168 EVENT_LOG_v1 {"time_micros": 1732347200383559, "cf_name": "default", "job": 29415, "event": "table_file_creation", "file_number": 266794, "file_size": 38810334, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3623676327, "largest_seqno": 4012775947, "table_properties": {"data_size": 32608496, "index_size": 1760448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25911340, "raw_average_key_size": 28, "raw_value_size": 108306414, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925405, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347197, "oldest_key_time": 0, "file_creation_time": 1732347197, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266794, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:33:20.423896 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266795.sst) /dev/shm/ToplingTemp/Topling-96235-BIykA2 2024/11/23-15:33:21.419104 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266795.sst): first pass time = 1.00's, 134.869'MB/sec 2024/11/23-15:33:22.753292 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266795.sst): old prealloc_size = 147639500, real_size = 35329328 2024/11/23-15:33:22.753872 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266795.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925307 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0333 GB avg-val = 117.05 avg-zval = 36.02 2024/11/23-15:33:22.754661 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266795.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:33:23.589114 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266795.sst): second pass time = 1.33's, 81.227'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 47.307'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3179.681'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 312.564'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35708.007'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.227'MB/sec zip pipeline throughput = 81.227'MB/sec entries = 925307 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.05 avg-zval = 36.02 usrkeys = 925307 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.05 avg-zval = 36.02 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925307 bytes = 3354256 } seq expand size = 6477149 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0333 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.4453 value = 3.2500 dict = 1.00 all = 3.3946 } Zip/UnZip{ index = 0.0957 value = 0.3077 dict = 1.00 all = 0.2946 } ---------------------------- total value len = 10.326940 GB avg = 0.160 KB (by entry num) total key len = 1.783211 GB avg = 0.028 KB total ukey len = 1.242442 GB avg = 0.016 KB total ukey num = 0.077778640 Billion total entry num = 0.064586369 Billion write speed all = 0.506883815 MB/sec (with seq num) write speed all = 0.485257142 MB/sec (without seq num) 2024/11/23-15:33:23.637600 139957634799168 [compaction_job.cc:1948] [default] [JOB 29415] Generated table #266795: 925307 keys, 39538688 bytes, temperature: kWarm 2024/11/23-15:33:23.637678 139957634799168 EVENT_LOG_v1 {"time_micros": 1732347203637643, "cf_name": "default", "job": 29415, "event": "table_file_creation", "file_number": 266795, "file_size": 39538688, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012776021, "table_properties": {"data_size": 33326112, "index_size": 1771712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25908596, "raw_average_key_size": 28, "raw_value_size": 108309218, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925307, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347200, "oldest_key_time": 0, "file_creation_time": 1732347200, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266795, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:33:23.675637 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266796.sst) /dev/shm/ToplingTemp/Topling-96235-Cntra3 2024/11/23-15:33:24.534393 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266796.sst): first pass time = 0.86's, 156.301'MB/sec 2024/11/23-15:33:25.864824 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266796.sst): old prealloc_size = 147639500, real_size = 35768048 2024/11/23-15:33:25.865371 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266796.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929589 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0338 GB avg-val = 116.38 avg-zval = 36.31 2024/11/23-15:33:25.866557 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266796.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:33:26.674355 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266796.sst): second pass time = 1.33's, 81.399'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 48.881'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3385.182'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.342'MB/sec write SST data time = 0.13's, 307.718'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 29584.010'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.399'MB/sec zip pipeline throughput = 81.399'MB/sec entries = 929589 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.38 avg-zval = 36.31 usrkeys = 929589 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.01 avg-zval = 36.31 TagRS{ kinds = RS_Key0_TagN bytes = 145360 } TagArray{ size = 746592 bytes = 2799728 } seq expand size = 5226144 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0338 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.4465 value = 3.2051 dict = 1.00 all = 3.3920 } Zip/UnZip{ index = 0.0957 value = 0.3120 dict = 1.00 all = 0.2948 } ---------------------------- total value len = 10.440356 GB avg = 0.159 KB (by entry num) total key len = 1.809239 GB avg = 0.028 KB total ukey len = 1.261034 GB avg = 0.016 KB total ukey num = 0.078708229 Billion total entry num = 0.065515958 Billion write speed all = 0.512654195 MB/sec (with seq num) write speed all = 0.490719082 MB/sec (without seq num) 2024/11/23-15:33:26.721533 139957634799168 [compaction_job.cc:1948] [default] [JOB 29415] Generated table #266796: 929589 keys, 39569308 bytes, temperature: kWarm 2024/11/23-15:33:26.721604 139957634799168 EVENT_LOG_v1 {"time_micros": 1732347206721569, "cf_name": "default", "job": 29415, "event": "table_file_creation", "file_number": 266796, "file_size": 39569308, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012774830, "table_properties": {"data_size": 33755744, "index_size": 1779712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26028492, "raw_average_key_size": 28, "raw_value_size": 108189326, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929589, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347203, "oldest_key_time": 0, "file_creation_time": 1732347203, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266796, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:33:26.770521 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266797.sst) /dev/shm/ToplingTemp/Topling-96235-HEWLN2 2024/11/23-15:33:27.789430 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266797.sst): first pass time = 1.02's, 131.732'MB/sec 2024/11/23-15:33:29.109499 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266797.sst): old prealloc_size = 147639500, real_size = 33143408 2024/11/23-15:33:29.110069 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266797.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928905 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.49 avg-zval = 33.53 2024/11/23-15:33:29.112408 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266797.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:33:29.966416 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266797.sst): second pass time = 1.32's, 82.018'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.604'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3234.993'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 290.885'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35973.412'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.018'MB/sec zip pipeline throughput = 82.018'MB/sec entries = 928905 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.49 avg-zval = 33.53 usrkeys = 928905 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.49 avg-zval = 33.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928905 bytes = 3367296 } seq expand size = 6502335 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5141 value = 3.4745 dict = 1.00 all = 3.5922 } Zip/UnZip{ index = 0.0951 value = 0.2878 dict = 1.00 all = 0.2784 } ---------------------------- total value len = 10.555067 GB avg = 0.159 KB (by entry num) total key len = 1.835249 GB avg = 0.028 KB total ukey len = 1.279612 GB avg = 0.016 KB total ukey num = 0.079637134 Billion total entry num = 0.066444863 Billion write speed all = 0.518472000 MB/sec (with seq num) write speed all = 0.496228950 MB/sec (without seq num) 2024/11/23-15:33:30.011747 139957634799168 [compaction_job.cc:1948] [default] [JOB 29415] Generated table #266797: 928905 keys, 37364074 bytes, temperature: kWarm 2024/11/23-15:33:30.011816 139957634799168 EVENT_LOG_v1 {"time_micros": 1732347210011781, "cf_name": "default", "job": 29415, "event": "table_file_creation", "file_number": 266797, "file_size": 37364074, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776210, "table_properties": {"data_size": 31144032, "index_size": 1766976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26009340, "raw_average_key_size": 28, "raw_value_size": 108208490, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928905, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347206, "oldest_key_time": 0, "file_creation_time": 1732347206, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266797, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:33:30.056492 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266798.sst) /dev/shm/ToplingTemp/Topling-96235-1CFLb2 2024/11/23-15:33:31.024408 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266798.sst): first pass time = 0.97's, 138.673'MB/sec 2024/11/23-15:33:32.308362 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266798.sst): old prealloc_size = 147639500, real_size = 32225472 2024/11/23-15:33:32.308887 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266798.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 935327 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0302 GB avg-val = 115.50 avg-zval = 32.30 2024/11/23-15:33:32.310531 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266798.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:33:33.170351 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266798.sst): second pass time = 1.28's, 84.184'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.305'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3514.322'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 284.711'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34069.667'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.184'MB/sec zip pipeline throughput = 84.184'MB/sec entries = 935327 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.50 avg-zval = 32.30 usrkeys = 935327 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.50 avg-zval = 32.30 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 935327 bytes = 3390576 } seq expand size = 6547289 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5125 value = 3.5757 dict = 1.00 all = 3.6806 } Zip/UnZip{ index = 0.0951 value = 0.2797 dict = 1.00 all = 0.2717 } ---------------------------- total value len = 10.669642 GB avg = 0.158 KB (by entry num) total key len = 1.861438 GB avg = 0.028 KB total ukey len = 1.298319 GB avg = 0.016 KB total ukey num = 0.080572461 Billion total entry num = 0.067380190 Billion write speed all = 0.524292017 MB/sec (with seq num) write speed all = 0.501738880 MB/sec (without seq num) 2024/11/23-15:33:33.215012 139957634799168 [compaction_job.cc:1948] [default] [JOB 29415] Generated table #266798: 935327 keys, 36465940 bytes, temperature: kWarm 2024/11/23-15:33:33.215082 139957634799168 EVENT_LOG_v1 {"time_micros": 1732347213215046, "cf_name": "default", "job": 29415, "event": "table_file_creation", "file_number": 266798, "file_size": 36465940, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776248, "table_properties": {"data_size": 30212016, "index_size": 1779456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26189156, "raw_average_key_size": 28, "raw_value_size": 108028654, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 935327, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347210, "oldest_key_time": 0, "file_creation_time": 1732347210, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266798, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:33:33.251129 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266799.sst) /dev/shm/ToplingTemp/Topling-96235-UaD3T1 2024/11/23-15:33:34.305847 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266799.sst): first pass time = 1.05's, 127.259'MB/sec 2024/11/23-15:33:35.625857 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266799.sst): old prealloc_size = 147639500, real_size = 35095360 2024/11/23-15:33:35.626456 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266799.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930505 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0331 GB avg-val = 116.24 avg-zval = 35.55 2024/11/23-15:33:35.627280 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266799.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:33:36.462538 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266799.sst): second pass time = 1.32's, 81.985'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 47.135'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3136.105'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.329'MB/sec write SST data time = 0.13's, 307.548'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 30801.429'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.985'MB/sec zip pipeline throughput = 81.985'MB/sec entries = 930505 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.24 avg-zval = 35.55 usrkeys = 930505 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.24 avg-zval = 35.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930505 bytes = 3489408 } seq expand size = 6513535 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0331 GB dict = 1.08 MB all = 0.0394 GB } UnZip/Zip{ index = 10.4440 value = 3.2697 dict = 1.00 all = 3.4034 } Zip/UnZip{ index = 0.0957 value = 0.3058 dict = 1.00 all = 0.2938 } ---------------------------- total value len = 10.784320 GB avg = 0.158 KB (by entry num) total key len = 1.887492 GB avg = 0.028 KB total ukey len = 1.316929 GB avg = 0.016 KB total ukey num = 0.081502966 Billion total entry num = 0.068310695 Billion write speed all = 0.530107102 MB/sec (with seq num) write speed all = 0.507245661 MB/sec (without seq num) 2024/11/23-15:33:36.510106 139957634799168 [compaction_job.cc:1948] [default] [JOB 29415] Generated table #266799: 930505 keys, 39436146 bytes, temperature: kWarm 2024/11/23-15:33:36.510178 139957634799168 EVENT_LOG_v1 {"time_micros": 1732347216510140, "cf_name": "default", "job": 29415, "event": "table_file_creation", "file_number": 266799, "file_size": 39436146, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776268, "table_properties": {"data_size": 33080688, "index_size": 1781888, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26054140, "raw_average_key_size": 28, "raw_value_size": 108163634, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930505, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347213, "oldest_key_time": 0, "file_creation_time": 1732347213, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266799, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:33:36.559528 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266800.sst) /dev/shm/ToplingTemp/Topling-96235-6Uhqq4 2024/11/23-15:33:37.551147 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266800.sst): first pass time = 0.99's, 135.358'MB/sec 2024/11/23-15:33:38.831073 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266800.sst): old prealloc_size = 147639500, real_size = 30284816 2024/11/23-15:33:38.831627 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266800.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931242 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0283 GB avg-val = 116.13 avg-zval = 30.37 2024/11/23-15:33:38.833918 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266800.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:33:39.652791 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266800.sst): second pass time = 1.28's, 84.546'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.001'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3343.503'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 269.719'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 28416.043'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.546'MB/sec zip pipeline throughput = 84.546'MB/sec entries = 931242 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.13 avg-zval = 30.37 usrkeys = 931242 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.13 avg-zval = 30.37 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931242 bytes = 3375760 } seq expand size = 6518694 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0283 GB dict = 1.08 MB all = 0.0345 GB } UnZip/Zip{ index = 10.5123 value = 3.8240 dict = 1.00 all = 3.8890 } Zip/UnZip{ index = 0.0951 value = 0.2615 dict = 1.00 all = 0.2571 } ---------------------------- total value len = 10.898981 GB avg = 0.157 KB (by entry num) total key len = 1.913567 GB avg = 0.028 KB total ukey len = 1.335553 GB avg = 0.016 KB total ukey num = 0.082434208 Billion total entry num = 0.069241937 Billion write speed all = 0.535923085 MB/sec (with seq num) write speed all = 0.512753079 MB/sec (without seq num) 2024/11/23-15:33:39.696011 139957634799168 [compaction_job.cc:1948] [default] [JOB 29415] Generated table #266800: 931242 keys, 34511948 bytes, temperature: kWarm 2024/11/23-15:33:39.696080 139957634799168 EVENT_LOG_v1 {"time_micros": 1732347219696044, "cf_name": "default", "job": 29415, "event": "table_file_creation", "file_number": 266800, "file_size": 34511948, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776300, "table_properties": {"data_size": 28280128, "index_size": 1771712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26074776, "raw_average_key_size": 28, "raw_value_size": 108142972, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931242, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347216, "oldest_key_time": 0, "file_creation_time": 1732347216, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266800, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:33:39.737001 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266801.sst) /dev/shm/ToplingTemp/Topling-96235-JAhsl3 2024/11/23-15:33:40.722207 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266801.sst): first pass time = 0.99's, 136.238'MB/sec 2024/11/23-15:33:42.053926 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266801.sst): old prealloc_size = 147639500, real_size = 35229632 2024/11/23-15:33:42.054501 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266801.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922697 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0332 GB avg-val = 117.46 avg-zval = 36.03 2024/11/23-15:33:42.055315 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266801.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:33:42.894968 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266801.sst): second pass time = 1.33's, 81.436'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 47.024'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3175.190'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.323'MB/sec write SST data time = 0.13's, 312.734'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 28465.230'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.436'MB/sec zip pipeline throughput = 81.436'MB/sec entries = 922697 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.46 avg-zval = 36.03 usrkeys = 922697 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.46 avg-zval = 36.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922697 bytes = 3460128 } seq expand size = 6458879 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.5135 value = 3.2603 dict = 1.00 all = 3.3938 } Zip/UnZip{ index = 0.0951 value = 0.3067 dict = 1.00 all = 0.2947 } ---------------------------- total value len = 11.013822 GB avg = 0.157 KB (by entry num) total key len = 1.939402 GB avg = 0.028 KB total ukey len = 1.354007 GB avg = 0.016 KB total ukey num = 0.083356905 Billion total entry num = 0.070164634 Billion write speed all = 0.541733841 MB/sec (with seq num) write speed all = 0.518258262 MB/sec (without seq num) 2024/11/23-15:33:42.944702 139957634799168 [compaction_job.cc:1948] [default] [JOB 29415] Generated table #266801: 922697 keys, 39548014 bytes, temperature: kWarm 2024/11/23-15:33:42.944773 139957634799168 EVENT_LOG_v1 {"time_micros": 1732347222944737, "cf_name": "default", "job": 29415, "event": "table_file_creation", "file_number": 266801, "file_size": 39548014, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776316, "table_properties": {"data_size": 33243504, "index_size": 1755264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25835516, "raw_average_key_size": 28, "raw_value_size": 108382282, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922697, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347219, "oldest_key_time": 0, "file_creation_time": 1732347219, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266801, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:33:42.982752 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266802.sst) /dev/shm/ToplingTemp/Topling-96235-6j0qK1 2024/11/23-15:33:43.949620 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266802.sst): first pass time = 0.97's, 138.822'MB/sec 2024/11/23-15:33:45.242309 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266802.sst): old prealloc_size = 147639500, real_size = 32165440 2024/11/23-15:33:45.242848 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266802.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931083 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0301 GB avg-val = 116.15 avg-zval = 32.38 2024/11/23-15:33:45.243676 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266802.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:33:46.085578 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266802.sst): second pass time = 1.29's, 83.710'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.296'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3442.235'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.325'MB/sec write SST data time = 0.13's, 286.911'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36148.823'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.710'MB/sec zip pipeline throughput = 83.710'MB/sec entries = 931083 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.15 avg-zval = 32.38 usrkeys = 931083 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.15 avg-zval = 32.38 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931083 bytes = 3491584 } seq expand size = 6517581 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.4438 value = 3.5871 dict = 1.00 all = 3.6763 } Zip/UnZip{ index = 0.0958 value = 0.2788 dict = 1.00 all = 0.2720 } ---------------------------- total value len = 11.128488 GB avg = 0.157 KB (by entry num) total key len = 1.965473 GB avg = 0.028 KB total ukey len = 1.372629 GB avg = 0.016 KB total ukey num = 0.084287988 Billion total entry num = 0.071095717 Billion write speed all = 0.547546659 MB/sec (with seq num) write speed all = 0.523762733 MB/sec (without seq num) 2024/11/23-15:33:46.130474 139957634799168 [compaction_job.cc:1948] [default] [JOB 29415] Generated table #266802: 931083 keys, 36508752 bytes, temperature: kWarm 2024/11/23-15:33:46.130544 139957634799168 EVENT_LOG_v1 {"time_micros": 1732347226130509, "cf_name": "default", "job": 29415, "event": "table_file_creation", "file_number": 266802, "file_size": 36508752, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012776344, "table_properties": {"data_size": 30149424, "index_size": 1783040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26070324, "raw_average_key_size": 28, "raw_value_size": 108147522, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931083, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347222, "oldest_key_time": 0, "file_creation_time": 1732347222, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266802, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:33:46.170309 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266803.sst) /dev/shm/ToplingTemp/Topling-96235-djk3S1 2024/11/23-15:33:47.179299 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266803.sst): first pass time = 1.01's, 133.028'MB/sec 2024/11/23-15:33:48.462556 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266803.sst): old prealloc_size = 147639500, real_size = 32280560 2024/11/23-15:33:48.463115 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266803.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925987 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0303 GB avg-val = 116.95 avg-zval = 32.70 2024/11/23-15:33:48.463849 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266803.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:33:49.288739 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266803.sst): second pass time = 1.28's, 84.435'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.754'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3306.536'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.333'MB/sec write SST data time = 0.13's, 279.092'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33494.917'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.435'MB/sec zip pipeline throughput = 84.435'MB/sec entries = 925987 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.95 avg-zval = 32.70 usrkeys = 925987 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.92 avg-zval = 32.70 TagRS{ kinds = RS_Key0_TagN bytes = 144800 } TagArray{ size = 789685 bytes = 2862624 } seq expand size = 5527795 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.4436 value = 3.5768 dict = 1.00 all = 3.7130 } Zip/UnZip{ index = 0.0958 value = 0.2796 dict = 1.00 all = 0.2693 } ---------------------------- total value len = 11.242306 GB avg = 0.156 KB (by entry num) total key len = 1.991400 GB avg = 0.028 KB total ukey len = 1.391149 GB avg = 0.016 KB total ukey num = 0.085213975 Billion total entry num = 0.072021704 Billion write speed all = 0.553316250 MB/sec (with seq num) write speed all = 0.529225777 MB/sec (without seq num) 2024/11/23-15:33:49.332931 139957634799168 [compaction_job.cc:1948] [default] [JOB 29415] Generated table #266803: 925987 keys, 36147640 bytes, temperature: kWarm 2024/11/23-15:33:49.333014 139957634799168 EVENT_LOG_v1 {"time_micros": 1732347229332963, "cf_name": "default", "job": 29415, "event": "table_file_creation", "file_number": 266803, "file_size": 36147640, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 30275552, "index_size": 1773312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25927636, "raw_average_key_size": 28, "raw_value_size": 108290194, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925987, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347226, "oldest_key_time": 0, "file_creation_time": 1732347226, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266803, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:33:49.377213 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266804.sst) /dev/shm/ToplingTemp/Topling-96235-G9j3L0 2024/11/23-15:33:50.297755 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266804.sst): first pass time = 0.92's, 145.809'MB/sec 2024/11/23-15:33:51.800329 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266804.sst): old prealloc_size = 147639500, real_size = 36871328 2024/11/23-15:33:51.800859 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266804.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926405 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0349 GB avg-val = 116.88 avg-zval = 37.65 2024/11/23-15:33:51.801096 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266804.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:33:52.512246 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266804.sst): second pass time = 1.50's, 72.100'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.71's, 53.798'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3462.868'MB/sec (index lex order gen) rebuild zvType time = 0.59's, 0.395'MB/sec write SST data time = 0.12's, 306.692'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34336.321'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 72.100'MB/sec zip pipeline throughput = 72.100'MB/sec entries = 926405 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.88 avg-zval = 37.65 usrkeys = 926405 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.91 avg-zval = 37.65 TagRS{ kinds = RS_Key0_TagN bytes = 144840 } TagArray{ size = 135719 bytes = 424144 } seq expand size = 950033 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0349 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.5132 value = 3.1046 dict = 1.00 all = 3.5045 } Zip/UnZip{ index = 0.0951 value = 0.3221 dict = 1.00 all = 0.2853 } ---------------------------- total value len = 11.351534 GB avg = 0.156 KB (by entry num) total key len = 2.017340 GB avg = 0.028 KB total ukey len = 1.409677 GB avg = 0.016 KB total ukey num = 0.086140380 Billion total entry num = 0.072948109 Billion write speed all = 0.558892445 MB/sec (with seq num) write speed all = 0.534495387 MB/sec (without seq num) 2024/11/23-15:33:52.558320 139957634799168 [compaction_job.cc:1948] [default] [JOB 29415] Generated table #266804: 926405 keys, 38298485 bytes, temperature: kWarm 2024/11/23-15:33:52.558391 139957634799168 EVENT_LOG_v1 {"time_micros": 1732347232558355, "cf_name": "default", "job": 29415, "event": "table_file_creation", "file_number": 266804, "file_size": 38298485, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700812, "table_properties": {"data_size": 34877200, "index_size": 1762368, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25939340, "raw_average_key_size": 28, "raw_value_size": 108278482, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926405, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347229, "oldest_key_time": 0, "file_creation_time": 1732347229, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266804, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:33:52.595028 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266805.sst) /dev/shm/ToplingTemp/Topling-96235-SJYyV1 2024/11/23-15:33:53.604403 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266805.sst): first pass time = 1.01's, 132.976'MB/sec 2024/11/23-15:33:54.959350 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266805.sst): old prealloc_size = 147639500, real_size = 36446512 2024/11/23-15:33:54.959922 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266805.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924293 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0344 GB avg-val = 117.21 avg-zval = 37.27 2024/11/23-15:33:54.960507 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266805.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:33:55.735133 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266805.sst): second pass time = 1.35's, 80.005'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.78's, 50.934'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3217.600'MB/sec (index lex order gen) rebuild zvType time = 0.65's, 0.357'MB/sec write SST data time = 0.13's, 310.053'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 35480.714'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.005'MB/sec zip pipeline throughput = 80.005'MB/sec entries = 924293 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.21 avg-zval = 37.27 usrkeys = 924293 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.37 avg-zval = 37.27 TagRS{ kinds = RS_Key0_TagN bytes = 144520 } TagArray{ size = 549688 bytes = 2061344 } seq expand size = 3847816 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.4456 value = 3.1452 dict = 1.00 all = 3.3968 } Zip/UnZip{ index = 0.0957 value = 0.3179 dict = 1.00 all = 0.2944 } ---------------------------- total value len = 11.463719 GB avg = 0.155 KB (by entry num) total key len = 2.043220 GB avg = 0.028 KB total ukey len = 1.428163 GB avg = 0.016 KB total ukey num = 0.087064673 Billion total entry num = 0.073872402 Billion write speed all = 0.564588278 MB/sec (with seq num) write speed all = 0.539885425 MB/sec (without seq num) 2024/11/23-15:33:55.782745 139957634799168 [compaction_job.cc:1948] [default] [JOB 29415] Generated table #266805: 924293 keys, 39512664 bytes, temperature: kWarm 2024/11/23-15:33:55.782816 139957634799168 EVENT_LOG_v1 {"time_micros": 1732347235782780, "cf_name": "default", "job": 29415, "event": "table_file_creation", "file_number": 266805, "file_size": 39512664, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4071354963, "table_properties": {"data_size": 34445536, "index_size": 1769728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25880204, "raw_average_key_size": 28, "raw_value_size": 108337618, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924293, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347232, "oldest_key_time": 0, "file_creation_time": 1732347232, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266805, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:33:55.820529 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266806.sst) /dev/shm/ToplingTemp/Topling-96235-2cLLn3 2024/11/23-15:33:56.727822 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266806.sst): first pass time = 0.91's, 147.939'MB/sec 2024/11/23-15:33:58.055537 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266806.sst): old prealloc_size = 147639500, real_size = 36506720 2024/11/23-15:33:58.056157 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266806.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925481 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 117.02 avg-zval = 37.29 2024/11/23-15:33:58.057468 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266806.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:33:58.876661 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266806.sst): second pass time = 1.33's, 81.618'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 49.166'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2945.618'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 311.407'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 28739.427'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.618'MB/sec zip pipeline throughput = 81.618'MB/sec entries = 925481 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.02 avg-zval = 37.29 usrkeys = 925481 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.02 avg-zval = 37.29 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925481 bytes = 3007824 } seq expand size = 6478367 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0404 GB } UnZip/Zip{ index = 10.5103 value = 3.1380 dict = 1.00 all = 3.3247 } Zip/UnZip{ index = 0.0951 value = 0.3187 dict = 1.00 all = 0.3008 } ---------------------------- total value len = 11.578502 GB avg = 0.155 KB (by entry num) total key len = 2.069133 GB avg = 0.028 KB total ukey len = 1.446672 GB avg = 0.016 KB total ukey num = 0.087990154 Billion total entry num = 0.074797883 Billion write speed all = 0.570394460 MB/sec (with seq num) write speed all = 0.545385411 MB/sec (without seq num) 2024/11/23-15:33:58.931778 139957634799168 [compaction_job.cc:1948] [default] [JOB 29415] Generated table #266806: 925481 keys, 40369406 bytes, temperature: kWarm 2024/11/23-15:33:58.931848 139957634799168 EVENT_LOG_v1 {"time_micros": 1732347238931812, "cf_name": "default", "job": 29415, "event": "table_file_creation", "file_number": 266806, "file_size": 40369406, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4031991524, "largest_seqno": 4071355639, "table_properties": {"data_size": 34514064, "index_size": 1761088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25913468, "raw_average_key_size": 28, "raw_value_size": 108304386, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925481, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347235, "oldest_key_time": 0, "file_creation_time": 1732347235, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266806, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:33:58.968898 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266807.sst) /dev/shm/ToplingTemp/Topling-96235-wJQh22 2024/11/23-15:33:59.044684 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266807.sst): first pass time = 0.08's, 233.340'MB/sec 2024/11/23-15:33:59.205144 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266807.sst): old prealloc_size = 147639500, real_size = 5318064 2024/11/23-15:33:59.205274 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266807.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 122346 prefix = 4 raw-key = 0.0020 GB zip-key = 0.0002 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0142 GB zip-val = 0.0051 GB avg-val = 116.47 avg-zval = 41.30 2024/11/23-15:33:59.205404 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266807.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.018 GB 2024/11/23-15:33:59.310946 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00/266807.sst): second pass time = 0.16's, 89.179'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.11's, 54.882'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2190.756'MB/sec (index lex order gen) rebuild zvType time = 0.09's, 0.347'MB/sec write SST data time = 0.02's, 332.937'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.149'MB, throughput = 6382.964'MB/sec zip my value time = 0.16's, unzip length = 0.014'GB zip my value throughput = 89.179'MB/sec zip pipeline throughput = 89.179'MB/sec entries = 122346 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.47 avg-zval = 41.30 usrkeys = 122346 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.47 avg-zval = 41.30 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 122346 bytes = 367056 } seq expand size = 856422 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0024 GB value = 0.0142 GB dict = 0.15 MB all = 0.0177 GB } __ZipSize{ index = 0.0002 GB value = 0.0051 GB dict = 0.15 MB all = 0.0058 GB } UnZip/Zip{ index = 10.4462 value = 2.8201 dict = 1.00 all = 3.0446 } Zip/UnZip{ index = 0.0957 value = 0.3546 dict = 1.00 all = 0.3285 } ---------------------------- total value len = 11.593609 GB avg = 0.155 KB (by entry num) total key len = 2.072559 GB avg = 0.028 KB total ukey len = 1.449119 GB avg = 0.016 KB total ukey num = 0.088112500 Billion total entry num = 0.074920229 Billion write speed all = 0.571158628 MB/sec (with seq num) write speed all = 0.546109127 MB/sec (without seq num) 2024/11/23-15:33:59.324806 139957634799168 [compaction_job.cc:1948] [default] [JOB 29415] Generated table #266807: 122346 keys, 5805607 bytes, temperature: kWarm 2024/11/23-15:33:59.324866 139957634799168 EVENT_LOG_v1 {"time_micros": 1732347239324830, "cf_name": "default", "job": 29415, "event": "table_file_creation", "file_number": 266807, "file_size": 5805607, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4054697261, "largest_seqno": 4071354276, "table_properties": {"data_size": 5053024, "index_size": 234240, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3425688, "raw_average_key_size": 28, "raw_value_size": 14249932, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 122346, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347238, "oldest_key_time": 0, "file_creation_time": 1732347238, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266807, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:33:59.335861 139957634799168 [compaction_job.cc:703] job-29415: subcompact[0], size: 0.916673 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29415/att-00 [266783,266784,266785,266786,266787,266788,266789,266790,266791,266792,266793,266794,266795,266796,266797,266798,266799,266800,266801,266802,266803,266804,266805,266806,266807] 2024/11/23-15:33:59.480748 139957634799168 (Original Log Time 2024/11/23-15:32:35.148478) EVENT_LOG_v1 {"time_micros": 1732347155148402, "job": 29415, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [266756], "files_L3": [266703, 266704, 266705, 266706, 266707, 266708, 266709, 266710, 266711, 266712, 266713, 266714, 266715, 266716, 266717, 266718, 266719, 266720, 266721, 266722, 266723, 266724, 266725, 266778], "score": 0.992886, "input_data_size": 937534158, "oldest_snapshot_seqno": 4071495652} 2024/11/23-15:33:59.480752 139957634799168 (Original Log Time 2024/11/23-15:33:59.479123) [compaction_job.cc:2022] [default] [JOB 29415] Compacted 1@2 + 24@3 files to L3 => 916673072 bytes 2024/11/23-15:33:59.480753 139957634799168 (Original Log Time 2024/11/23-15:33:59.480676) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 25 0 0 0] max score 0.03, MB/sec: 11.1 rd, 10.9 wr, level 3, files in(1, 24) out(25 +0 blob) MB in(27.2, 866.9 +0.0 blob) out(874.2 +0.0 blob), read-write-amplify(65.1) write-amplify(32.2) OK, records in: 22674175, records dropped: 112110 output_compression: Snappy 2024/11/23-15:33:59.480757 139957634799168 (Original Log Time 2024/11/23-15:33:59.480721) EVENT_LOG_v1 {"time_micros": 1732347239480700, "cf": "default", "job": 29415, "event": "compaction_finished", "compaction_time_micros": 84187388, "compaction_time_cpu_micros": 35860616, "output_level": 3, "num_output_files": 25, "total_output_size": 916673072, "num_input_records": 22674175, "num_output_records": 22562065, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 646674898, "file_range_sync_nanos": 0, "file_fsync_nanos": 18261, "file_prepare_write_nanos": 65226, "lsm_state": [0, 0, 0, 25, 0, 0, 0]} 2024/11/23-15:33:59.618454 139957634799168 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732347154,"executesMs":84489,"compactionJobId":29415,"attempt":0,"compactionInputRawBytes":3254822743,"compactionInputZipBytes":937534158,"compactionOutputRawBytes":0,"compactionOutputZipBytes":916673072,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-0c71235a4dbfc83671a4f448c263d0cf-3a5577fb53c516ce-00"}}, response = 2024/11/23-15:33:59.618524 139957634799168 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-29415/att-00: olev 3, work 84.490 s, result 5.977 ms, install 1.649 ms, input{raw 3.031 GiB zip 894.102 MiB}