2024/11/23-15:17:51.942092 139957349447232 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00: fork to child time = 0.023319 sec 2024/11/23-15:17:51.943050 139957349447232 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00 2024/11/23-15:17:51.951858 139957349447232 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00 2024/11/23-15:17:51.953075 139957349447232 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-15:17:51.953436 139957349447232 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-15:17:52.006505 139957349447232 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-29064/att-00/MANIFEST-206347 2024/11/23-15:17:52.043245 139957349447232 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-29064/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 266025, last_sequence is 4060934491, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 265863 2024/11/23-15:17:52.043256 139957349447232 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-15:17:52.043447 139957349447232 [version_set.cc:5721] Creating manifest 266025 2024/11/23-15:17:52.078153 139957349447232 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-15:17:52.103031 139957349447232 [compaction_job.cc:2437] [default] [JOB 29064] Compacting 1@2 + 24@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-15:17:52.103062 139957349447232 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 21727 Base level 2, inputs: [265999(25MB)], [265940(39MB) 265941(38MB) 265942(34MB) 265943(35MB) 265944(34MB) 265945(35MB) 265946(34MB) 265947(36MB) 265948(34MB) 265949(34MB) 265950(34MB) 265951(35MB) 265952(38MB) 265953(36MB) 265954(35MB) 265955(34MB) 265956(36MB) 265957(34MB) 265958(37MB) 265959(32MB) 265960(33MB) 265961(36MB) 265962(37MB) 266021(38MB)] 2024/11/23-15:17:52.113320 139957349447232 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-15:17:52.113361 139957349447232 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-15:17:52.113390 139957349447232 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-15:17:52.113400 139957349447232 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-15:17:52.113417 139957349447232 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-15:17:52.759935 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266026.sst) /dev/shm/ToplingTemp/Topling-91282-m1pW4t 2024/11/23-15:17:53.684318 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266026.sst): first pass time = 0.92's, 145.207'MB/sec 2024/11/23-15:17:56.166840 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266026.sst): old prealloc_size = 147639500, real_size = 37313872 2024/11/23-15:17:56.167388 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266026.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927832 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0353 GB avg-val = 116.66 avg-zval = 38.06 2024/11/23-15:17:56.168184 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266026.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:17:57.530720 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266026.sst): second pass time = 2.48's, 43.618'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.36's, 30.196'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3354.104'MB/sec (index lex order gen) rebuild zvType time = 1.09's, 0.212'MB/sec write SST data time = 0.27's, 152.557'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 37062.393'MB/sec zip my value time = 2.48's, unzip length = 0.108'GB zip my value throughput = 43.618'MB/sec zip pipeline throughput = 75.984'MB/sec entries = 927832 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.66 avg-zval = 38.06 usrkeys = 927832 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.66 avg-zval = 38.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927832 bytes = 3015472 } seq expand size = 6494824 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0353 GB dict = 1.08 MB all = 0.0412 GB } UnZip/Zip{ index = 10.5164 value = 3.0648 dict = 1.00 all = 3.2591 } Zip/UnZip{ index = 0.0951 value = 0.3263 dict = 1.00 all = 0.3068 } ---------------------------- total value len = 8.963353 GB avg = 0.168 KB (by entry num) total key len = 1.466800 GB avg = 0.028 KB total ukey len = 1.016435 GB avg = 0.015 KB total ukey num = 0.066478267 Billion total entry num = 0.053285996 Billion write speed all = 0.454169823 MB/sec (with seq num) write speed all = 0.435607571 MB/sec (without seq num) 2024/11/23-15:17:57.586420 139957349447232 [compaction_job.cc:1948] [default] [JOB 29064] Generated table #266026: 927832 keys, 41183108 bytes, temperature: kWarm 2024/11/23-15:17:57.586549 139957349447232 EVENT_LOG_v1 {"time_micros": 1732346277586468, "cf_name": "default", "job": 29064, "event": "table_file_creation", "file_number": 266026, "file_size": 41183108, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4012777911, "largest_seqno": 4060760199, "table_properties": {"data_size": 35317184, "index_size": 1764544, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25979296, "raw_average_key_size": 28, "raw_value_size": 108238532, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927832, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346272, "oldest_key_time": 0, "file_creation_time": 1732346272, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266026, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:17:57.638110 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266027.sst) /dev/shm/ToplingTemp/Topling-91282-mLMPKv 2024/11/23-15:17:58.669253 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266027.sst): first pass time = 1.03's, 130.171'MB/sec 2024/11/23-15:18:00.394641 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266027.sst): old prealloc_size = 147639500, real_size = 36393696 2024/11/23-15:18:00.395191 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266027.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928913 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0344 GB avg-val = 116.49 avg-zval = 37.03 2024/11/23-15:18:00.396005 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266027.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:18:01.510706 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266027.sst): second pass time = 1.72's, 62.749'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.12's, 36.394'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3353.942'MB/sec (index lex order gen) rebuild zvType time = 0.99's, 0.235'MB/sec write SST data time = 0.13's, 316.752'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35743.958'MB/sec zip my value time = 1.72's, unzip length = 0.108'GB zip my value throughput = 62.749'MB/sec zip pipeline throughput = 62.749'MB/sec entries = 928913 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.49 avg-zval = 37.03 usrkeys = 928913 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.49 avg-zval = 37.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928913 bytes = 3367328 } seq expand size = 6502391 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0406 GB } UnZip/Zip{ index = 10.5123 value = 3.1461 dict = 1.00 all = 3.3044 } Zip/UnZip{ index = 0.0951 value = 0.3178 dict = 1.00 all = 0.3026 } ---------------------------- total value len = 9.078064 GB avg = 0.167 KB (by entry num) total key len = 1.492810 GB avg = 0.028 KB total ukey len = 1.035013 GB avg = 0.015 KB total ukey num = 0.067407180 Billion total entry num = 0.054214909 Billion write speed all = 0.460217577 MB/sec (with seq num) write speed all = 0.441335010 MB/sec (without seq num) 2024/11/23-15:18:01.560938 139957349447232 [compaction_job.cc:1948] [default] [JOB 29064] Generated table #266027: 928913 keys, 40618066 bytes, temperature: kWarm 2024/11/23-15:18:01.561024 139957349447232 EVENT_LOG_v1 {"time_micros": 1732346281560972, "cf_name": "default", "job": 29064, "event": "table_file_creation", "file_number": 266027, "file_size": 40618066, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 4060760058, "table_properties": {"data_size": 34394000, "index_size": 1767296, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26009564, "raw_average_key_size": 28, "raw_value_size": 108208298, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928913, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346277, "oldest_key_time": 0, "file_creation_time": 1732346277, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266027, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:18:01.598062 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266028.sst) /dev/shm/ToplingTemp/Topling-91282-bYiPsw 2024/11/23-15:18:02.855775 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266028.sst): first pass time = 1.26's, 106.719'MB/sec 2024/11/23-15:18:04.151444 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266028.sst): old prealloc_size = 147639500, real_size = 30298672 2024/11/23-15:18:04.151973 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266028.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931622 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0283 GB avg-val = 116.07 avg-zval = 30.36 2024/11/23-15:18:04.152805 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266028.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:18:05.222709 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266028.sst): second pass time = 1.29's, 83.560'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.07's, 32.339'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3380.181'MB/sec (index lex order gen) rebuild zvType time = 0.93's, 0.251'MB/sec write SST data time = 0.14's, 244.885'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 30394.146'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.560'MB/sec zip pipeline throughput = 83.560'MB/sec entries = 931622 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.07 avg-zval = 30.36 usrkeys = 931622 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.07 avg-zval = 30.36 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931622 bytes = 3493600 } seq expand size = 6521354 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0283 GB dict = 1.08 MB all = 0.0346 GB } UnZip/Zip{ index = 10.4476 value = 3.8233 dict = 1.00 all = 3.8743 } Zip/UnZip{ index = 0.0957 value = 0.2616 dict = 1.00 all = 0.2581 } ---------------------------- total value len = 9.192718 GB avg = 0.167 KB (by entry num) total key len = 1.518895 GB avg = 0.028 KB total ukey len = 1.053645 GB avg = 0.015 KB total ukey num = 0.068338802 Billion total entry num = 0.055146531 Billion write speed all = 0.466269499 MB/sec (with seq num) write speed all = 0.447065560 MB/sec (without seq num) 2024/11/23-15:18:05.265525 139957349447232 [compaction_job.cc:1948] [default] [JOB 29064] Generated table #266028: 931622 keys, 34643098 bytes, temperature: kWarm 2024/11/23-15:18:05.265593 139957349447232 EVENT_LOG_v1 {"time_micros": 1732346285265559, "cf_name": "default", "job": 29064, "event": "table_file_creation", "file_number": 266028, "file_size": 34643098, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 4012776133, "table_properties": {"data_size": 28282144, "index_size": 1783424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26085416, "raw_average_key_size": 28, "raw_value_size": 108132412, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931622, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346281, "oldest_key_time": 0, "file_creation_time": 1732346281, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266028, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:18:05.305431 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266029.sst) /dev/shm/ToplingTemp/Topling-91282-0PJRCx 2024/11/23-15:18:06.502568 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266029.sst): first pass time = 1.20's, 112.119'MB/sec 2024/11/23-15:18:07.934989 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266029.sst): old prealloc_size = 147639500, real_size = 33002160 2024/11/23-15:18:07.935700 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266029.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928076 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.62 avg-zval = 33.39 2024/11/23-15:18:07.936599 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266029.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:18:08.898143 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266029.sst): second pass time = 1.43's, 75.612'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 38.765'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2577.268'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.287'MB/sec write SST data time = 0.15's, 244.027'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35208.295'MB/sec zip my value time = 1.43's, unzip length = 0.108'GB zip my value throughput = 75.612'MB/sec zip pipeline throughput = 75.612'MB/sec entries = 928076 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.62 avg-zval = 33.39 usrkeys = 928076 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.62 avg-zval = 33.39 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928076 bytes = 3480304 } seq expand size = 6496532 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4449 value = 3.4922 dict = 1.00 all = 3.5949 } Zip/UnZip{ index = 0.0957 value = 0.2864 dict = 1.00 all = 0.2782 } ---------------------------- total value len = 9.307446 GB avg = 0.166 KB (by entry num) total key len = 1.544881 GB avg = 0.028 KB total ukey len = 1.072207 GB avg = 0.015 KB total ukey num = 0.069266878 Billion total entry num = 0.056074607 Billion write speed all = 0.472319139 MB/sec (with seq num) write speed all = 0.452795135 MB/sec (without seq num) 2024/11/23-15:18:08.944296 139957349447232 [compaction_job.cc:1948] [default] [JOB 29064] Generated table #266029: 928076 keys, 37336040 bytes, temperature: kWarm 2024/11/23-15:18:08.944358 139957349447232 EVENT_LOG_v1 {"time_micros": 1732346288944325, "cf_name": "default", "job": 29064, "event": "table_file_creation", "file_number": 266029, "file_size": 37336040, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3377687296, "largest_seqno": 4012776149, "table_properties": {"data_size": 30992864, "index_size": 1777088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25986128, "raw_average_key_size": 28, "raw_value_size": 108231736, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928076, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346285, "oldest_key_time": 0, "file_creation_time": 1732346285, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266029, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:18:08.980482 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266030.sst) /dev/shm/ToplingTemp/Topling-91282-My446x 2024/11/23-15:18:10.100787 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266030.sst): first pass time = 1.12's, 119.808'MB/sec 2024/11/23-15:18:11.597042 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266030.sst): old prealloc_size = 147639500, real_size = 32809616 2024/11/23-15:18:11.597629 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266030.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931746 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 116.05 avg-zval = 33.06 2024/11/23-15:18:11.598408 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266030.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:18:12.486110 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266030.sst): second pass time = 1.50's, 72.322'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 41.530'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3146.092'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.306'MB/sec write SST data time = 0.13's, 288.564'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 37716.755'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 72.322'MB/sec zip pipeline throughput = 72.322'MB/sec entries = 931746 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.05 avg-zval = 33.06 usrkeys = 931746 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.05 avg-zval = 33.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931746 bytes = 3261120 } seq expand size = 6522222 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5123 value = 3.5102 dict = 1.00 all = 3.6352 } Zip/UnZip{ index = 0.0951 value = 0.2849 dict = 1.00 all = 0.2751 } ---------------------------- total value len = 9.422097 GB avg = 0.165 KB (by entry num) total key len = 1.570970 GB avg = 0.028 KB total ukey len = 1.090842 GB avg = 0.016 KB total ukey num = 0.070198624 Billion total entry num = 0.057006353 Billion write speed all = 0.478369777 MB/sec (with seq num) write speed all = 0.458524458 MB/sec (without seq num) 2024/11/23-15:18:12.534970 139957349447232 [compaction_job.cc:1948] [default] [JOB 29064] Generated table #266030: 931746 keys, 36921722 bytes, temperature: kWarm 2024/11/23-15:18:12.535036 139957349447232 EVENT_LOG_v1 {"time_micros": 1732346292535005, "cf_name": "default", "job": 29064, "event": "table_file_creation", "file_number": 266030, "file_size": 36921722, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3759280178, "largest_seqno": 4012776176, "table_properties": {"data_size": 30803840, "index_size": 1772672, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26088888, "raw_average_key_size": 28, "raw_value_size": 108128872, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931746, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346288, "oldest_key_time": 0, "file_creation_time": 1732346288, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266030, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:18:12.572893 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266031.sst) /dev/shm/ToplingTemp/Topling-91282-rurdfv 2024/11/23-15:18:13.736105 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266031.sst): first pass time = 1.16's, 115.389'MB/sec 2024/11/23-15:18:15.550951 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266031.sst): old prealloc_size = 147639500, real_size = 32802144 2024/11/23-15:18:15.551510 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266031.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931885 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 116.03 avg-zval = 33.05 2024/11/23-15:18:15.552302 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266031.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:18:16.557219 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266031.sst): second pass time = 1.81's, 59.615'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.01's, 36.686'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3388.184'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.302'MB/sec write SST data time = 0.23's, 158.366'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33718.290'MB/sec zip my value time = 1.81's, unzip length = 0.108'GB zip my value throughput = 59.615'MB/sec zip pipeline throughput = 132.189'MB/sec entries = 931885 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.03 avg-zval = 33.05 usrkeys = 931885 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.03 avg-zval = 33.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931885 bytes = 3261616 } seq expand size = 6523195 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5162 value = 3.5109 dict = 1.00 all = 3.6359 } Zip/UnZip{ index = 0.0951 value = 0.2848 dict = 1.00 all = 0.2750 } ---------------------------- total value len = 9.536745 GB avg = 0.165 KB (by entry num) total key len = 1.597063 GB avg = 0.028 KB total ukey len = 1.109479 GB avg = 0.016 KB total ukey num = 0.071130509 Billion total entry num = 0.057938238 Billion write speed all = 0.484408391 MB/sec (with seq num) write speed all = 0.464242232 MB/sec (without seq num) 2024/11/23-15:18:16.606259 139957349447232 [compaction_job.cc:1948] [default] [JOB 29064] Generated table #266031: 931885 keys, 36914264 bytes, temperature: kWarm 2024/11/23-15:18:16.606348 139957349447232 EVENT_LOG_v1 {"time_micros": 1732346296606302, "cf_name": "default", "job": 29064, "event": "table_file_creation", "file_number": 266031, "file_size": 36914264, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3801926538, "largest_seqno": 4012776192, "table_properties": {"data_size": 30796688, "index_size": 1772288, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26092780, "raw_average_key_size": 28, "raw_value_size": 108125074, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931885, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346292, "oldest_key_time": 0, "file_creation_time": 1732346292, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266031, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:18:16.679620 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266032.sst) /dev/shm/ToplingTemp/Topling-91282-NFQTMx 2024/11/23-15:18:17.708015 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266032.sst): first pass time = 1.03's, 130.519'MB/sec 2024/11/23-15:18:19.285544 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266032.sst): old prealloc_size = 147639500, real_size = 33738432 2024/11/23-15:18:19.286194 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266032.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1019217 prefix = 4 raw-key = 0.0163 GB zip-key = 0.0020 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1057 GB zip-val = 0.0315 GB avg-val = 103.69 avg-zval = 30.94 2024/11/23-15:18:19.286624 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266032.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0025 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:18:20.280198 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266032.sst): second pass time = 1.58's, 67.029'MB/sec, value only(78.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.99's, 36.238'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3112.923'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.339'MB/sec write SST data time = 0.24's, 149.241'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.056'MB, throughput = 32059.343'MB/sec zip my value time = 1.58's, unzip length = 0.106'GB zip my value throughput = 67.029'MB/sec zip pipeline throughput = 102.771'MB/sec entries = 1019217 avg-key = 28.00 avg-zkey = 1.91 avg-val = 103.69 avg-zval = 30.94 usrkeys = 1019217 avg-key = 20.00 avg-zkey = 1.91 avg-val = 106.40 avg-zval = 30.94 TagRS{ kinds = RS_Key0_TagN bytes = 159360 } TagArray{ size = 395130 bytes = 1333584 } seq expand size = 2765910 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0204 GB value = 0.1057 GB dict = 1.06 MB all = 0.1342 GB } __ZipSize{ index = 0.0020 GB value = 0.0315 GB dict = 1.06 MB all = 0.0360 GB } UnZip/Zip{ index = 10.4449 value = 3.3515 dict = 1.00 all = 3.7239 } Zip/UnZip{ index = 0.0957 value = 0.2984 dict = 1.00 all = 0.2685 } ---------------------------- total value len = 9.645191 GB avg = 0.164 KB (by entry num) total key len = 1.625601 GB avg = 0.028 KB total ukey len = 1.129864 GB avg = 0.016 KB total ukey num = 0.072149726 Billion total entry num = 0.058957455 Billion write speed all = 0.490288842 MB/sec (with seq num) write speed all = 0.469771255 MB/sec (without seq num) 2024/11/23-15:18:20.326990 139957349447232 [compaction_job.cc:1948] [default] [JOB 29064] Generated table #266032: 1019217 keys, 36042577 bytes, temperature: kWarm 2024/11/23-15:18:20.327085 139957349447232 EVENT_LOG_v1 {"time_micros": 1732346300327032, "cf_name": "default", "job": 29064, "event": "table_file_creation", "file_number": 266032, "file_size": 36042577, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012775259, "table_properties": {"data_size": 31531824, "index_size": 1951616, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 28538076, "raw_average_key_size": 28, "raw_value_size": 105679653, "raw_average_value_size": 103, "num_data_blocks": 1, "num_entries": 1019217, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346296, "oldest_key_time": 0, "file_creation_time": 1732346296, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266032, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:18:20.367163 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266033.sst) /dev/shm/ToplingTemp/Topling-91282-EwvmUw 2024/11/23-15:18:21.284744 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266033.sst): first pass time = 0.92's, 146.279'MB/sec 2024/11/23-15:18:22.880776 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266033.sst): old prealloc_size = 147639500, real_size = 36089072 2024/11/23-15:18:22.881380 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266033.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 995029 prefix = 4 raw-key = 0.0159 GB zip-key = 0.0019 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1064 GB zip-val = 0.0339 GB avg-val = 106.89 avg-zval = 34.12 2024/11/23-15:18:22.881699 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266033.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0025 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:18:23.809546 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266033.sst): second pass time = 1.60's, 66.667'MB/sec, value only(79.2% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.771'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3244.072'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.314'MB/sec write SST data time = 0.14's, 278.831'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.052'MB, throughput = 37485.334'MB/sec zip my value time = 1.60's, unzip length = 0.106'GB zip my value throughput = 66.667'MB/sec zip pipeline throughput = 66.667'MB/sec entries = 995029 avg-key = 28.00 avg-zkey = 1.90 avg-val = 106.89 avg-zval = 34.12 usrkeys = 995029 avg-key = 20.00 avg-zkey = 1.90 avg-val = 108.51 avg-zval = 34.12 TagRS{ kinds = RS_Key0_TagN bytes = 155560 } TagArray{ size = 230713 bytes = 807504 } seq expand size = 1614991 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0199 GB value = 0.1064 GB dict = 1.05 MB all = 0.1342 GB } __ZipSize{ index = 0.0019 GB value = 0.0339 GB dict = 1.05 MB all = 0.0379 GB } UnZip/Zip{ index = 10.5149 value = 3.1330 dict = 1.00 all = 3.5446 } Zip/UnZip{ index = 0.0951 value = 0.3192 dict = 1.00 all = 0.2821 } ---------------------------- total value len = 9.753163 GB avg = 0.163 KB (by entry num) total key len = 1.653462 GB avg = 0.028 KB total ukey len = 1.149764 GB avg = 0.016 KB total ukey num = 0.073144755 Billion total entry num = 0.059952484 Billion write speed all = 0.496121511 MB/sec (with seq num) write speed all = 0.475260850 MB/sec (without seq num) 2024/11/23-15:18:23.868144 139957349447232 [compaction_job.cc:1948] [default] [JOB 29064] Generated table #266033: 995029 keys, 37865512 bytes, temperature: kWarm 2024/11/23-15:18:23.868204 139957349447232 EVENT_LOG_v1 {"time_micros": 1732346303868170, "cf_name": "default", "job": 29064, "event": "table_file_creation", "file_number": 266033, "file_size": 37865512, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3425813787, "table_properties": {"data_size": 33947552, "index_size": 1892608, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 27860812, "raw_average_key_size": 28, "raw_value_size": 106356972, "raw_average_value_size": 106, "num_data_blocks": 1, "num_entries": 995029, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346300, "oldest_key_time": 0, "file_creation_time": 1732346300, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266033, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:18:23.916176 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266034.sst) /dev/shm/ToplingTemp/Topling-91282-JRfMdu 2024/11/23-15:18:24.840082 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266034.sst): first pass time = 0.92's, 145.279'MB/sec 2024/11/23-15:18:26.365842 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266034.sst): old prealloc_size = 147639500, real_size = 30972000 2024/11/23-15:18:26.366447 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266034.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931429 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0290 GB avg-val = 116.10 avg-zval = 31.10 2024/11/23-15:18:26.367301 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266034.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:18:27.196059 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266034.sst): second pass time = 1.53's, 70.909'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.539'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3057.027'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.13's, 280.239'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36850.718'MB/sec zip my value time = 1.53's, unzip length = 0.108'GB zip my value throughput = 70.909'MB/sec zip pipeline throughput = 70.909'MB/sec entries = 931429 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.10 avg-zval = 31.10 usrkeys = 931429 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.10 avg-zval = 31.10 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931429 bytes = 3492880 } seq expand size = 6520003 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0290 GB dict = 1.08 MB all = 0.0353 GB } UnZip/Zip{ index = 10.5145 value = 3.7331 dict = 1.00 all = 3.8005 } Zip/UnZip{ index = 0.0951 value = 0.2679 dict = 1.00 all = 0.2631 } ---------------------------- total value len = 9.867821 GB avg = 0.162 KB (by entry num) total key len = 1.679542 GB avg = 0.028 KB total ukey len = 1.168393 GB avg = 0.016 KB total ukey num = 0.074076184 Billion total entry num = 0.060883913 Billion write speed all = 0.502168816 MB/sec (with seq num) write speed all = 0.480987181 MB/sec (without seq num) 2024/11/23-15:18:27.240713 139957349447232 [compaction_job.cc:1948] [default] [JOB 29064] Generated table #266034: 931429 keys, 35316092 bytes, temperature: kWarm 2024/11/23-15:18:27.240781 139957349447232 EVENT_LOG_v1 {"time_micros": 1732346307240747, "cf_name": "default", "job": 29064, "event": "table_file_creation", "file_number": 266034, "file_size": 35316092, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3425814902, "largest_seqno": 4012775885, "table_properties": {"data_size": 28967248, "index_size": 1771712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26080012, "raw_average_key_size": 28, "raw_value_size": 108137762, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346303, "oldest_key_time": 0, "file_creation_time": 1732346303, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266034, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:18:27.277836 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266035.sst) /dev/shm/ToplingTemp/Topling-91282-GjWhgw 2024/11/23-15:18:28.112735 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266035.sst): first pass time = 0.83's, 160.766'MB/sec 2024/11/23-15:18:29.405074 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266035.sst): old prealloc_size = 147639500, real_size = 30794656 2024/11/23-15:18:29.405600 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266035.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931309 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0288 GB avg-val = 116.12 avg-zval = 30.91 2024/11/23-15:18:29.407037 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266035.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:18:30.233544 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266035.sst): second pass time = 1.29's, 83.755'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.134'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3441.649'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.333'MB/sec write SST data time = 0.13's, 270.983'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32704.719'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.755'MB/sec zip pipeline throughput = 83.755'MB/sec entries = 931309 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.12 avg-zval = 30.91 usrkeys = 931309 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.12 avg-zval = 30.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931309 bytes = 3259600 } seq expand size = 6519163 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0288 GB dict = 1.08 MB all = 0.0349 GB } UnZip/Zip{ index = 10.5123 value = 3.7562 dict = 1.00 all = 3.8452 } Zip/UnZip{ index = 0.0951 value = 0.2662 dict = 1.00 all = 0.2601 } ---------------------------- total value len = 9.982481 GB avg = 0.161 KB (by entry num) total key len = 1.705619 GB avg = 0.028 KB total ukey len = 1.187019 GB avg = 0.016 KB total ukey num = 0.075007493 Billion total entry num = 0.061815222 Billion write speed all = 0.508222017 MB/sec (with seq num) write speed all = 0.486719218 MB/sec (without seq num) 2024/11/23-15:18:30.277662 139957349447232 [compaction_job.cc:1948] [default] [JOB 29064] Generated table #266035: 931309 keys, 34905556 bytes, temperature: kWarm 2024/11/23-15:18:30.277731 139957349447232 EVENT_LOG_v1 {"time_micros": 1732346310277695, "cf_name": "default", "job": 29064, "event": "table_file_creation", "file_number": 266035, "file_size": 34905556, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3805383762, "largest_seqno": 4012775902, "table_properties": {"data_size": 28789776, "index_size": 1771840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26076652, "raw_average_key_size": 28, "raw_value_size": 108141150, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931309, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346307, "oldest_key_time": 0, "file_creation_time": 1732346307, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266035, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:18:30.314953 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266036.sst) /dev/shm/ToplingTemp/Topling-91282-Z0x9ju 2024/11/23-15:18:31.285091 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266036.sst): first pass time = 0.97's, 138.358'MB/sec 2024/11/23-15:18:32.588016 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266036.sst): old prealloc_size = 147639500, real_size = 31199520 2024/11/23-15:18:32.588590 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266036.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931462 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0292 GB avg-val = 116.09 avg-zval = 31.34 2024/11/23-15:18:32.590725 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266036.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:18:33.422365 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266036.sst): second pass time = 1.30's, 83.048'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.601'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3229.937'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.332'MB/sec write SST data time = 0.13's, 269.127'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34097.401'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.048'MB/sec zip pipeline throughput = 83.048'MB/sec entries = 931462 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.09 avg-zval = 31.34 usrkeys = 931462 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.09 avg-zval = 31.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931462 bytes = 3492992 } seq expand size = 6520234 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.5122 value = 3.7040 dict = 1.00 all = 3.7761 } Zip/UnZip{ index = 0.0951 value = 0.2700 dict = 1.00 all = 0.2648 } ---------------------------- total value len = 10.097138 GB avg = 0.161 KB (by entry num) total key len = 1.731700 GB avg = 0.028 KB total ukey len = 1.205648 GB avg = 0.016 KB total ukey num = 0.075938955 Billion total entry num = 0.062746684 Billion write speed all = 0.514270282 MB/sec (with seq num) write speed all = 0.492446494 MB/sec (without seq num) 2024/11/23-15:18:33.465665 139957349447232 [compaction_job.cc:1948] [default] [JOB 29064] Generated table #266036: 931462 keys, 35543716 bytes, temperature: kWarm 2024/11/23-15:18:33.465740 139957349447232 EVENT_LOG_v1 {"time_micros": 1732346313465698, "cf_name": "default", "job": 29064, "event": "table_file_creation", "file_number": 266036, "file_size": 35543716, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 4012775918, "table_properties": {"data_size": 29194320, "index_size": 1772160, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26080936, "raw_average_key_size": 28, "raw_value_size": 108136860, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931462, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346310, "oldest_key_time": 0, "file_creation_time": 1732346310, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266036, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:18:33.502828 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266037.sst) /dev/shm/ToplingTemp/Topling-91282-KRDJuu 2024/11/23-15:18:34.541478 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266037.sst): first pass time = 1.04's, 129.227'MB/sec 2024/11/23-15:18:35.872548 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266037.sst): old prealloc_size = 147639500, real_size = 32261504 2024/11/23-15:18:35.873154 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266037.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924660 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0303 GB avg-val = 117.15 avg-zval = 32.74 2024/11/23-15:18:35.874619 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266037.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:18:36.701135 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266037.sst): second pass time = 1.33's, 81.433'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 44.015'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3005.290'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 287.534'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 35665.611'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.433'MB/sec zip pipeline throughput = 81.433'MB/sec entries = 924660 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.15 avg-zval = 32.74 usrkeys = 924660 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.15 avg-zval = 32.74 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924660 bytes = 3351904 } seq expand size = 6472620 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5140 value = 3.5786 dict = 1.00 all = 3.6803 } Zip/UnZip{ index = 0.0951 value = 0.2794 dict = 1.00 all = 0.2717 } ---------------------------- total value len = 10.211938 GB avg = 0.160 KB (by entry num) total key len = 1.757590 GB avg = 0.028 KB total ukey len = 1.224142 GB avg = 0.016 KB total ukey num = 0.076863615 Billion total entry num = 0.063671344 Billion write speed all = 0.520312769 MB/sec (with seq num) write speed all = 0.498170533 MB/sec (without seq num) 2024/11/23-15:18:36.744843 139957349447232 [compaction_job.cc:1948] [default] [JOB 29064] Generated table #266037: 924660 keys, 36469572 bytes, temperature: kWarm 2024/11/23-15:18:36.744913 139957349447232 EVENT_LOG_v1 {"time_micros": 1732346316744878, "cf_name": "default", "job": 29064, "event": "table_file_creation", "file_number": 266037, "file_size": 36469572, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3609689375, "largest_seqno": 4012775947, "table_properties": {"data_size": 30271216, "index_size": 1758912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25890480, "raw_average_key_size": 28, "raw_value_size": 108327384, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924660, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346313, "oldest_key_time": 0, "file_creation_time": 1732346313, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266037, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:18:36.783256 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266038.sst) /dev/shm/ToplingTemp/Topling-91282-U4CLXu 2024/11/23-15:18:37.644321 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266038.sst): first pass time = 0.86's, 155.882'MB/sec 2024/11/23-15:18:38.984759 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266038.sst): old prealloc_size = 147639500, real_size = 35785248 2024/11/23-15:18:38.985357 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266038.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924897 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0338 GB avg-val = 117.12 avg-zval = 36.51 2024/11/23-15:18:38.986886 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266038.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:18:39.833233 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266038.sst): second pass time = 1.34's, 80.889'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 47.137'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3086.699'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 311.310'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35146.804'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.889'MB/sec zip pipeline throughput = 80.889'MB/sec entries = 924897 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.12 avg-zval = 36.51 usrkeys = 924897 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.12 avg-zval = 36.51 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924897 bytes = 3352768 } seq expand size = 6474279 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0338 GB dict = 1.08 MB all = 0.0400 GB } UnZip/Zip{ index = 10.3762 value = 3.2075 dict = 1.00 all = 3.3560 } Zip/UnZip{ index = 0.0964 value = 0.3118 dict = 1.00 all = 0.2980 } ---------------------------- total value len = 10.326733 GB avg = 0.160 KB (by entry num) total key len = 1.783487 GB avg = 0.028 KB total ukey len = 1.242640 GB avg = 0.016 KB total ukey num = 0.077788512 Billion total entry num = 0.064596241 Billion write speed all = 0.526356959 MB/sec (with seq num) write speed all = 0.503896141 MB/sec (without seq num) 2024/11/23-15:18:39.885479 139957349447232 [compaction_job.cc:1948] [default] [JOB 29064] Generated table #266038: 924897 keys, 39993778 bytes, temperature: kWarm 2024/11/23-15:18:39.885551 139957349447232 EVENT_LOG_v1 {"time_micros": 1732346319885515, "cf_name": "default", "job": 29064, "event": "table_file_creation", "file_number": 266038, "file_size": 39993778, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775967, "table_properties": {"data_size": 33771088, "index_size": 1782720, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25897116, "raw_average_key_size": 28, "raw_value_size": 108320698, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924897, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346316, "oldest_key_time": 0, "file_creation_time": 1732346316, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266038, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:18:39.931808 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266039.sst) /dev/shm/ToplingTemp/Topling-91282-OKIQrv 2024/11/23-15:18:40.901440 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266039.sst): first pass time = 0.97's, 138.428'MB/sec 2024/11/23-15:18:42.253926 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266039.sst): old prealloc_size = 147639500, real_size = 33523824 2024/11/23-15:18:42.254502 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266039.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929869 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0315 GB avg-val = 116.34 avg-zval = 33.89 2024/11/23-15:18:42.256638 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266039.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:18:43.103339 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266039.sst): second pass time = 1.35's, 80.035'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.425'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3253.960'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.13's, 284.980'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35889.916'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.035'MB/sec zip pipeline throughput = 80.035'MB/sec entries = 929869 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.34 avg-zval = 33.89 usrkeys = 929869 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.78 avg-zval = 33.89 TagRS{ kinds = RS_Key0_TagN bytes = 145400 } TagArray{ size = 855623 bytes = 3208608 } seq expand size = 5989361 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.4459 value = 3.4332 dict = 1.00 all = 3.5569 } Zip/UnZip{ index = 0.0957 value = 0.2913 dict = 1.00 all = 0.2811 } ---------------------------- total value len = 10.440904 GB avg = 0.159 KB (by entry num) total key len = 1.809524 GB avg = 0.028 KB total ukey len = 1.261237 GB avg = 0.016 KB total ukey num = 0.078718381 Billion total entry num = 0.065526110 Billion write speed all = 0.532375236 MB/sec (with seq num) write speed all = 0.509594331 MB/sec (without seq num) 2024/11/23-15:18:43.148940 139957349447232 [compaction_job.cc:1948] [default] [JOB 29064] Generated table #266039: 929869 keys, 37734066 bytes, temperature: kWarm 2024/11/23-15:18:43.149030 139957349447232 EVENT_LOG_v1 {"time_micros": 1732346323148968, "cf_name": "default", "job": 29064, "event": "table_file_creation", "file_number": 266039, "file_size": 37734066, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776021, "table_properties": {"data_size": 31510816, "index_size": 1780352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26036332, "raw_average_key_size": 28, "raw_value_size": 108181454, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929869, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346319, "oldest_key_time": 0, "file_creation_time": 1732346319, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266039, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:18:43.200391 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266040.sst) /dev/shm/ToplingTemp/Topling-91282-1bLb5t 2024/11/23-15:18:44.135043 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266040.sst): first pass time = 0.93's, 143.607'MB/sec 2024/11/23-15:18:45.458342 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266040.sst): old prealloc_size = 147639500, real_size = 32723040 2024/11/23-15:18:45.458919 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266040.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929780 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1082 GB zip-val = 0.0307 GB avg-val = 116.35 avg-zval = 33.05 2024/11/23-15:18:45.460886 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266040.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:18:46.287027 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266040.sst): second pass time = 1.32's, 81.808'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 44.413'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3168.652'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.333'MB/sec write SST data time = 0.13's, 282.196'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35865.178'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.808'MB/sec zip pipeline throughput = 81.808'MB/sec entries = 929780 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.35 avg-zval = 33.05 usrkeys = 929780 avg-key = 20.00 avg-zkey = 1.89 avg-val = 122.54 avg-zval = 33.05 TagRS{ kinds = RS_Key0_TagN bytes = 145360 } TagArray{ size = 821029 bytes = 3078880 } seq expand size = 5747203 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5838 value = 3.5201 dict = 1.00 all = 3.6469 } Zip/UnZip{ index = 0.0945 value = 0.2841 dict = 1.00 all = 0.2742 } ---------------------------- total value len = 10.554835 GB avg = 0.159 KB (by entry num) total key len = 1.835557 GB avg = 0.028 KB total ukey len = 1.279833 GB avg = 0.016 KB total ukey num = 0.079648161 Billion total entry num = 0.066455890 Billion write speed all = 0.538383301 MB/sec (with seq num) write speed all = 0.515282343 MB/sec (without seq num) 2024/11/23-15:18:46.333001 139957349447232 [compaction_job.cc:1948] [default] [JOB 29064] Generated table #266040: 929780 keys, 36803272 bytes, temperature: kWarm 2024/11/23-15:18:46.333075 139957349447232 EVENT_LOG_v1 {"time_micros": 1732346326333038, "cf_name": "default", "job": 29064, "event": "table_file_creation", "file_number": 266040, "file_size": 36803272, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776210, "table_properties": {"data_size": 30733392, "index_size": 1756992, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26033840, "raw_average_key_size": 28, "raw_value_size": 108183912, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929780, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346323, "oldest_key_time": 0, "file_creation_time": 1732346323, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266040, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:18:46.376243 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266041.sst) /dev/shm/ToplingTemp/Topling-91282-ciBYgy 2024/11/23-15:18:47.393919 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266041.sst): first pass time = 1.02's, 131.892'MB/sec 2024/11/23-15:18:48.687665 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266041.sst): old prealloc_size = 147639500, real_size = 31658768 2024/11/23-15:18:48.688245 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266041.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931692 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0297 GB avg-val = 116.06 avg-zval = 31.83 2024/11/23-15:18:48.690181 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266041.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:18:49.531128 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266041.sst): second pass time = 1.29's, 83.630'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.549'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3179.334'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 278.018'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 28969.423'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.630'MB/sec zip pipeline throughput = 83.630'MB/sec entries = 931692 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.06 avg-zval = 31.83 usrkeys = 931692 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.06 avg-zval = 31.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931692 bytes = 3377392 } seq expand size = 6521844 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.08 MB all = 0.0359 GB } UnZip/Zip{ index = 10.5140 value = 3.6465 dict = 1.00 all = 3.7400 } Zip/UnZip{ index = 0.0951 value = 0.2742 dict = 1.00 all = 0.2674 } ---------------------------- total value len = 10.669487 GB avg = 0.158 KB (by entry num) total key len = 1.861645 GB avg = 0.028 KB total ukey len = 1.298466 GB avg = 0.016 KB total ukey num = 0.080579853 Billion total entry num = 0.067387582 Billion write speed all = 0.544421931 MB/sec (with seq num) write speed all = 0.521000406 MB/sec (without seq num) 2024/11/23-15:18:49.575597 139957349447232 [compaction_job.cc:1948] [default] [JOB 29064] Generated table #266041: 931692 keys, 35887224 bytes, temperature: kWarm 2024/11/23-15:18:49.575675 139957349447232 EVENT_LOG_v1 {"time_micros": 1732346329575639, "cf_name": "default", "job": 29064, "event": "table_file_creation", "file_number": 266041, "file_size": 35887224, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776228, "table_properties": {"data_size": 29653376, "index_size": 1772288, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26087376, "raw_average_key_size": 28, "raw_value_size": 108130408, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931692, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346326, "oldest_key_time": 0, "file_creation_time": 1732346326, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266041, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:18:49.612232 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266042.sst) /dev/shm/ToplingTemp/Topling-91282-S0Ynwu 2024/11/23-15:18:50.656678 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266042.sst): first pass time = 1.04's, 128.512'MB/sec 2024/11/23-15:18:51.974079 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266042.sst): old prealloc_size = 147639500, real_size = 33223584 2024/11/23-15:18:51.974601 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266042.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933210 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0312 GB avg-val = 115.82 avg-zval = 33.44 2024/11/23-15:18:51.976700 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266042.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:18:52.834476 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266042.sst): second pass time = 1.32's, 82.100'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.536'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3510.745'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.13's, 288.663'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33601.396'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.100'MB/sec zip pipeline throughput = 82.100'MB/sec entries = 933210 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.82 avg-zval = 33.44 usrkeys = 933210 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.82 avg-zval = 33.44 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933210 bytes = 3382896 } seq expand size = 6532470 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4444 value = 3.4640 dict = 1.00 all = 3.5832 } Zip/UnZip{ index = 0.0957 value = 0.2887 dict = 1.00 all = 0.2791 } ---------------------------- total value len = 10.784107 GB avg = 0.158 KB (by entry num) total key len = 1.887775 GB avg = 0.028 KB total ukey len = 1.317131 GB avg = 0.016 KB total ukey num = 0.081513063 Billion total entry num = 0.068320792 Billion write speed all = 0.550457901 MB/sec (with seq num) write speed all = 0.526715433 MB/sec (without seq num) 2024/11/23-15:18:52.880200 139957349447232 [compaction_job.cc:1948] [default] [JOB 29064] Generated table #266042: 933210 keys, 37457256 bytes, temperature: kWarm 2024/11/23-15:18:52.880271 139957349447232 EVENT_LOG_v1 {"time_micros": 1732346332880235, "cf_name": "default", "job": 29064, "event": "table_file_creation", "file_number": 266042, "file_size": 37457256, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3485447251, "largest_seqno": 4012776268, "table_properties": {"data_size": 31203088, "index_size": 1787008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26129880, "raw_average_key_size": 28, "raw_value_size": 108087848, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933210, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346329, "oldest_key_time": 0, "file_creation_time": 1732346329, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266042, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:18:52.924838 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266043.sst) /dev/shm/ToplingTemp/Topling-91282-CbI3ax 2024/11/23-15:18:53.946071 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266043.sst): first pass time = 1.02's, 131.433'MB/sec 2024/11/23-15:18:55.244527 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266043.sst): old prealloc_size = 147639500, real_size = 32837808 2024/11/23-15:18:55.245106 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266043.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924170 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0308 GB avg-val = 117.23 avg-zval = 33.37 2024/11/23-15:18:55.245947 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266043.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:18:56.081166 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266043.sst): second pass time = 1.30's, 83.516'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.417'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3179.289'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 293.294'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 36878.852'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.516'MB/sec zip pipeline throughput = 83.516'MB/sec entries = 924170 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.23 avg-zval = 33.37 usrkeys = 924170 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.23 avg-zval = 33.37 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924170 bytes = 3465648 } seq expand size = 6469190 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.4434 value = 3.5134 dict = 1.00 all = 3.6119 } Zip/UnZip{ index = 0.0958 value = 0.2846 dict = 1.00 all = 0.2769 } ---------------------------- total value len = 10.898918 GB avg = 0.157 KB (by entry num) total key len = 1.913651 GB avg = 0.028 KB total ukey len = 1.335614 GB avg = 0.016 KB total ukey num = 0.082437233 Billion total entry num = 0.069244962 Billion write speed all = 0.556490765 MB/sec (with seq num) write speed all = 0.532430527 MB/sec (without seq num) 2024/11/23-15:18:56.128032 139957349447232 [compaction_job.cc:1948] [default] [JOB 29064] Generated table #266043: 924170 keys, 37160102 bytes, temperature: kWarm 2024/11/23-15:18:56.128105 139957349447232 EVENT_LOG_v1 {"time_micros": 1732346336128068, "cf_name": "default", "job": 29064, "event": "table_file_creation", "file_number": 266043, "file_size": 37160102, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776300, "table_properties": {"data_size": 30836704, "index_size": 1769856, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25876760, "raw_average_key_size": 28, "raw_value_size": 108341056, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924170, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346332, "oldest_key_time": 0, "file_creation_time": 1732346332, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266043, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:18:56.162086 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266044.sst) /dev/shm/ToplingTemp/Topling-91282-NWCd4t 2024/11/23-15:18:57.214778 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266044.sst): first pass time = 1.05's, 127.504'MB/sec 2024/11/23-15:18:58.570318 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266044.sst): old prealloc_size = 147639500, real_size = 34419248 2024/11/23-15:18:58.570862 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266044.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929853 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0324 GB avg-val = 116.34 avg-zval = 34.85 2024/11/23-15:18:58.571714 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266044.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:18:59.420911 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266044.sst): second pass time = 1.35's, 79.888'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.431'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3423.695'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 298.786'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33778.634'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 79.888'MB/sec zip pipeline throughput = 79.888'MB/sec entries = 929853 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.34 avg-zval = 34.85 usrkeys = 929853 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.34 avg-zval = 34.85 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929853 bytes = 3370736 } seq expand size = 6508971 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0324 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.4457 value = 3.3383 dict = 1.00 all = 3.4734 } Zip/UnZip{ index = 0.0957 value = 0.2996 dict = 1.00 all = 0.2879 } ---------------------------- total value len = 11.013609 GB avg = 0.157 KB (by entry num) total key len = 1.939687 GB avg = 0.028 KB total ukey len = 1.354211 GB avg = 0.016 KB total ukey num = 0.083367086 Billion total entry num = 0.070174815 Billion write speed all = 0.562521379 MB/sec (with seq num) write speed all = 0.538141586 MB/sec (without seq num) 2024/11/23-15:18:59.468056 139957349447232 [compaction_job.cc:1948] [default] [JOB 29064] Generated table #266044: 929853 keys, 38642166 bytes, temperature: kWarm 2024/11/23-15:18:59.468129 139957349447232 EVENT_LOG_v1 {"time_micros": 1732346339468092, "cf_name": "default", "job": 29064, "event": "table_file_creation", "file_number": 266044, "file_size": 38642166, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776316, "table_properties": {"data_size": 32406240, "index_size": 1780352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26035884, "raw_average_key_size": 28, "raw_value_size": 108181886, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929853, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346336, "oldest_key_time": 0, "file_creation_time": 1732346336, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266044, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:18:59.508086 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266045.sst) /dev/shm/ToplingTemp/Topling-91282-XCXgXw 2024/11/23-15:19:00.492129 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266045.sst): first pass time = 0.98's, 136.401'MB/sec 2024/11/23-15:19:01.796627 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266045.sst): old prealloc_size = 147639500, real_size = 31527200 2024/11/23-15:19:01.797247 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266045.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929200 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0295 GB avg-val = 116.44 avg-zval = 31.78 2024/11/23-15:19:01.798400 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266045.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:19:02.634210 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266045.sst): second pass time = 1.30's, 83.024'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.820'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2990.068'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.328'MB/sec write SST data time = 0.13's, 279.400'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32982.214'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.024'MB/sec zip pipeline throughput = 83.024'MB/sec entries = 929200 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.44 avg-zval = 31.78 usrkeys = 929200 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.44 avg-zval = 31.78 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929200 bytes = 3484512 } seq expand size = 6504400 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.08 MB all = 0.0359 GB } UnZip/Zip{ index = 10.5121 value = 3.6645 dict = 1.00 all = 3.7424 } Zip/UnZip{ index = 0.0951 value = 0.2729 dict = 1.00 all = 0.2672 } ---------------------------- total value len = 11.128313 GB avg = 0.157 KB (by entry num) total key len = 1.965705 GB avg = 0.028 KB total ukey len = 1.372795 GB avg = 0.016 KB total ukey num = 0.084296286 Billion total entry num = 0.071104015 Billion write speed all = 0.568553167 MB/sec (with seq num) write speed all = 0.543854002 MB/sec (without seq num) 2024/11/23-15:19:02.679161 139957349447232 [compaction_job.cc:1948] [default] [JOB 29064] Generated table #266045: 929200 keys, 35864476 bytes, temperature: kWarm 2024/11/23-15:19:02.679234 139957349447232 EVENT_LOG_v1 {"time_micros": 1732346342679199, "cf_name": "default", "job": 29064, "event": "table_file_creation", "file_number": 266045, "file_size": 35864476, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776344, "table_properties": {"data_size": 29526864, "index_size": 1767872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26017600, "raw_average_key_size": 28, "raw_value_size": 108200148, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929200, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346339, "oldest_key_time": 0, "file_creation_time": 1732346339, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266045, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:19:02.715514 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266046.sst) /dev/shm/ToplingTemp/Topling-91282-hPvv1v 2024/11/23-15:19:03.725566 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266046.sst): first pass time = 1.01's, 132.890'MB/sec 2024/11/23-15:19:05.019024 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266046.sst): old prealloc_size = 147639500, real_size = 32467600 2024/11/23-15:19:05.019649 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266046.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923122 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1084 GB zip-val = 0.0305 GB avg-val = 117.40 avg-zval = 32.99 2024/11/23-15:19:05.021838 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266046.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:19:05.858733 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266046.sst): second pass time = 1.29's, 83.835'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.809'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2977.723'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.13's, 284.798'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 35084.646'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.835'MB/sec zip pipeline throughput = 83.835'MB/sec entries = 923122 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.40 avg-zval = 32.99 usrkeys = 923122 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.40 avg-zval = 32.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923122 bytes = 3461728 } seq expand size = 6461854 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.3750 value = 3.5581 dict = 1.00 all = 3.6485 } Zip/UnZip{ index = 0.0964 value = 0.2810 dict = 1.00 all = 0.2741 } ---------------------------- total value len = 11.243145 GB avg = 0.156 KB (by entry num) total key len = 1.991552 GB avg = 0.028 KB total ukey len = 1.391257 GB avg = 0.016 KB total ukey num = 0.085219408 Billion total entry num = 0.072027137 Billion write speed all = 0.574581149 MB/sec (with seq num) write speed all = 0.549564825 MB/sec (without seq num) 2024/11/23-15:19:05.876731 139957349447232 [compaction_job.cc:1948] [default] [JOB 29064] Generated table #266046: 923122 keys, 36787042 bytes, temperature: kWarm 2024/11/23-15:19:05.876792 139957349447232 EVENT_LOG_v1 {"time_micros": 1732346345876760, "cf_name": "default", "job": 29064, "event": "table_file_creation", "file_number": 266046, "file_size": 36787042, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012776360, "table_properties": {"data_size": 30457088, "index_size": 1779520, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25847416, "raw_average_key_size": 28, "raw_value_size": 108370436, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923122, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346342, "oldest_key_time": 0, "file_creation_time": 1732346342, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266046, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:19:05.916271 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266047.sst) /dev/shm/ToplingTemp/Topling-91282-QOUfTu 2024/11/23-15:19:06.914788 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266047.sst): first pass time = 1.00's, 134.422'MB/sec 2024/11/23-15:19:08.389437 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266047.sst): old prealloc_size = 147639500, real_size = 36282368 2024/11/23-15:19:08.389997 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266047.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930292 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0343 GB avg-val = 116.27 avg-zval = 36.85 2024/11/23-15:19:08.390467 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266047.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:19:09.113177 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266047.sst): second pass time = 1.47's, 73.417'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.72's, 52.243'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3261.141'MB/sec (index lex order gen) rebuild zvType time = 0.60's, 0.389'MB/sec write SST data time = 0.13's, 301.652'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35716.367'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.417'MB/sec zip pipeline throughput = 73.417'MB/sec entries = 930292 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.27 avg-zval = 36.85 usrkeys = 930292 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.54 avg-zval = 36.85 TagRS{ kinds = RS_Key0_TagN bytes = 145440 } TagArray{ size = 167920 bytes = 524768 } seq expand size = 1175440 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0343 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.5130 value = 3.1555 dict = 1.00 all = 3.5499 } Zip/UnZip{ index = 0.0951 value = 0.3169 dict = 1.00 all = 0.2817 } ---------------------------- total value len = 11.352490 GB avg = 0.156 KB (by entry num) total key len = 2.017600 GB avg = 0.028 KB total ukey len = 1.409863 GB avg = 0.016 KB total ukey num = 0.086149700 Billion total entry num = 0.072957429 Billion write speed all = 0.580377213 MB/sec (with seq num) write speed all = 0.555041362 MB/sec (without seq num) 2024/11/23-15:19:09.159719 139957349447232 [compaction_job.cc:1948] [default] [JOB 29064] Generated table #266047: 930292 keys, 37809117 bytes, temperature: kWarm 2024/11/23-15:19:09.159792 139957349447232 EVENT_LOG_v1 {"time_micros": 1732346349159756, "cf_name": "default", "job": 29064, "event": "table_file_creation", "file_number": 266047, "file_size": 37809117, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012773626, "table_properties": {"data_size": 34279792, "index_size": 1769792, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26048176, "raw_average_key_size": 28, "raw_value_size": 108169664, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930292, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346345, "oldest_key_time": 0, "file_creation_time": 1732346345, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266047, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:19:09.197614 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266048.sst) /dev/shm/ToplingTemp/Topling-91282-dTQYMv 2024/11/23-15:19:10.237100 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266048.sst): first pass time = 1.04's, 129.124'MB/sec 2024/11/23-15:19:11.694503 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266048.sst): old prealloc_size = 147639500, real_size = 37161872 2024/11/23-15:19:11.695054 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266048.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924727 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0351 GB avg-val = 117.14 avg-zval = 38.01 2024/11/23-15:19:11.696135 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266048.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:19:12.438620 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266048.sst): second pass time = 1.46's, 74.368'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.74's, 53.235'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3370.294'MB/sec (index lex order gen) rebuild zvType time = 0.62's, 0.375'MB/sec write SST data time = 0.13's, 312.538'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32096.921'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 74.368'MB/sec zip pipeline throughput = 74.368'MB/sec entries = 924727 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.14 avg-zval = 38.01 usrkeys = 924727 avg-key = 20.00 avg-zkey = 1.93 avg-val = 120.06 avg-zval = 38.01 TagRS{ kinds = RS_Key0_TagN bytes = 144600 } TagArray{ size = 385506 bytes = 1445664 } seq expand size = 2698542 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0351 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.3781 value = 3.0819 dict = 1.00 all = 3.3883 } Zip/UnZip{ index = 0.0964 value = 0.3245 dict = 1.00 all = 0.2951 } ---------------------------- total value len = 11.463514 GB avg = 0.155 KB (by entry num) total key len = 2.043493 GB avg = 0.028 KB total ukey len = 1.428358 GB avg = 0.016 KB total ukey num = 0.087074427 Billion total entry num = 0.073882156 Billion write speed all = 0.586235939 MB/sec (with seq num) write speed all = 0.560582662 MB/sec (without seq num) 2024/11/23-15:19:12.485803 139957349447232 [compaction_job.cc:1948] [default] [JOB 29064] Generated table #266048: 924727 keys, 39611567 bytes, temperature: kWarm 2024/11/23-15:19:12.485874 139957349447232 EVENT_LOG_v1 {"time_micros": 1732346352485839, "cf_name": "default", "job": 29064, "event": "table_file_creation", "file_number": 266048, "file_size": 39611567, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4057581901, "table_properties": {"data_size": 35148416, "index_size": 1782080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25892356, "raw_average_key_size": 28, "raw_value_size": 108325482, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924727, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346349, "oldest_key_time": 0, "file_creation_time": 1732346349, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266048, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:19:12.522405 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266049.sst) /dev/shm/ToplingTemp/Topling-91282-xGl5Jv 2024/11/23-15:19:13.600109 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266049.sst): first pass time = 1.08's, 124.544'MB/sec 2024/11/23-15:19:14.930671 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266049.sst): old prealloc_size = 147639500, real_size = 36747904 2024/11/23-15:19:14.931296 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266049.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927295 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0348 GB avg-val = 116.74 avg-zval = 37.48 2024/11/23-15:19:14.933186 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266049.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:19:15.747410 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266049.sst): second pass time = 1.33's, 81.405'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 49.731'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2936.310'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.338'MB/sec write SST data time = 0.13's, 312.557'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34418.339'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.405'MB/sec zip pipeline throughput = 81.405'MB/sec entries = 927295 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.74 avg-zval = 37.48 usrkeys = 927295 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.74 avg-zval = 37.48 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927295 bytes = 3013728 } seq expand size = 6491065 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0348 GB dict = 1.08 MB all = 0.0406 GB } UnZip/Zip{ index = 10.5134 value = 3.1150 dict = 1.00 all = 3.3046 } Zip/UnZip{ index = 0.0951 value = 0.3210 dict = 1.00 all = 0.3026 } ---------------------------- total value len = 11.578259 GB avg = 0.155 KB (by entry num) total key len = 2.069457 GB avg = 0.028 KB total ukey len = 1.446904 GB avg = 0.016 KB total ukey num = 0.088001722 Billion total entry num = 0.074809451 Billion write speed all = 0.592257983 MB/sec (with seq num) write speed all = 0.566286461 MB/sec (without seq num) 2024/11/23-15:19:15.796487 139957349447232 [compaction_job.cc:1948] [default] [JOB 29064] Generated table #266049: 927295 keys, 40615886 bytes, temperature: kWarm 2024/11/23-15:19:15.796557 139957349447232 EVENT_LOG_v1 {"time_micros": 1732346355796521, "cf_name": "default", "job": 29064, "event": "table_file_creation", "file_number": 266049, "file_size": 40615886, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4012776751, "largest_seqno": 4060760167, "table_properties": {"data_size": 34751856, "index_size": 1764032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25964260, "raw_average_key_size": 28, "raw_value_size": 108253566, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927295, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346352, "oldest_key_time": 0, "file_creation_time": 1732346352, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266049, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:19:15.832879 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266050.sst) /dev/shm/ToplingTemp/Topling-91282-ffMRIu 2024/11/23-15:19:15.903321 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266050.sst): first pass time = 0.07's, 234.010'MB/sec 2024/11/23-15:19:16.058821 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266050.sst): old prealloc_size = 147639500, real_size = 5014944 2024/11/23-15:19:16.058955 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266050.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 113825 prefix = 4 raw-key = 0.0018 GB zip-key = 0.0002 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.0133 GB zip-val = 0.0048 GB avg-val = 116.73 avg-zval = 41.87 2024/11/23-15:19:16.059134 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266050.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0002 GB, waited 0.000 sec, Key+Value = 0.016 GB 2024/11/23-15:19:16.156699 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00/266050.sst): second pass time = 0.15's, 85.913'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.10's, 56.018'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2014.360'MB/sec (index lex order gen) rebuild zvType time = 0.08's, 0.349'MB/sec write SST data time = 0.02's, 338.602'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.138'MB, throughput = 3646.556'MB/sec zip my value time = 0.15's, unzip length = 0.013'GB zip my value throughput = 85.913'MB/sec zip pipeline throughput = 85.913'MB/sec entries = 113825 avg-key = 28.00 avg-zkey = 1.94 avg-val = 116.73 avg-zval = 41.87 usrkeys = 113825 avg-key = 20.00 avg-zkey = 1.94 avg-val = 123.73 avg-zval = 41.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 113825 bytes = 355712 } seq expand size = 796775 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0023 GB value = 0.0133 GB dict = 0.14 MB all = 0.0165 GB } __ZipSize{ index = 0.0002 GB value = 0.0048 GB dict = 0.14 MB all = 0.0055 GB } UnZip/Zip{ index = 10.3102 value = 2.7882 dict = 1.00 all = 3.0051 } Zip/UnZip{ index = 0.0970 value = 0.3587 dict = 1.00 all = 0.3328 } ---------------------------- total value len = 11.592343 GB avg = 0.155 KB (by entry num) total key len = 2.072644 GB avg = 0.028 KB total ukey len = 1.449180 GB avg = 0.016 KB total ukey num = 0.088115547 Billion total entry num = 0.074923276 Billion write speed all = 0.592996951 MB/sec (with seq num) write speed all = 0.566986374 MB/sec (without seq num) 2024/11/23-15:19:16.169777 139957349447232 [compaction_job.cc:1948] [default] [JOB 29064] Generated table #266050: 113825 keys, 5482119 bytes, temperature: kWarm 2024/11/23-15:19:16.169836 139957349447232 EVENT_LOG_v1 {"time_micros": 1732346356169807, "cf_name": "default", "job": 29064, "event": "table_file_creation", "file_number": 266050, "file_size": 5482119, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4042068720, "largest_seqno": 4060759886, "table_properties": {"data_size": 4765520, "index_size": 220800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3187100, "raw_average_key_size": 28, "raw_value_size": 13287262, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 113825, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346355, "oldest_key_time": 0, "file_creation_time": 1732346355, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266050, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:19:16.183664 139957349447232 [compaction_job.cc:703] job-29064: subcompact[0], size: 0.903607 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29064/att-00 [266026,266027,266028,266029,266030,266031,266032,266033,266034,266035,266036,266037,266038,266039,266040,266041,266042,266043,266044,266045,266046,266047,266048,266049,266050] 2024/11/23-15:19:16.332715 139957349447232 (Original Log Time 2024/11/23-15:17:52.103140) EVENT_LOG_v1 {"time_micros": 1732346272103083, "job": 29064, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [265999], "files_L3": [265940, 265941, 265942, 265943, 265944, 265945, 265946, 265947, 265948, 265949, 265950, 265951, 265952, 265953, 265954, 265955, 265956, 265957, 265958, 265959, 265960, 265961, 265962, 266021], "score": 0.995624, "input_data_size": 928562409, "oldest_snapshot_seqno": 4060934491} 2024/11/23-15:19:16.332718 139957349447232 (Original Log Time 2024/11/23-15:19:16.331147) [compaction_job.cc:2022] [default] [JOB 29064] Compacted 1@2 + 24@3 files to L3 => 903607394 bytes 2024/11/23-15:19:16.332719 139957349447232 (Original Log Time 2024/11/23-15:19:16.332638) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 25 0 0 0] max score 0.03, MB/sec: 11.0 rd, 10.7 wr, level 3, files in(1, 24) out(25 +0 blob) MB in(25.7, 859.9 +0.0 blob) out(861.7 +0.0 blob), read-write-amplify(68.0) write-amplify(33.6) OK, records in: 22672634, records dropped: 107522 output_compression: Snappy 2024/11/23-15:19:16.332723 139957349447232 (Original Log Time 2024/11/23-15:19:16.332689) EVENT_LOG_v1 {"time_micros": 1732346356332663, "cf": "default", "job": 29064, "event": "compaction_finished", "compaction_time_micros": 84080531, "compaction_time_cpu_micros": 35565366, "output_level": 3, "num_output_files": 25, "total_output_size": 903607394, "num_input_records": 22672634, "num_output_records": 22565112, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 664550957, "file_range_sync_nanos": 0, "file_fsync_nanos": 19764, "file_prepare_write_nanos": 79205, "lsm_state": [0, 0, 0, 25, 0, 0, 0]} 2024/11/23-15:19:16.463591 139957349447232 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732346271,"executesMs":84381,"compactionJobId":29064,"attempt":0,"compactionInputRawBytes":3252969785,"compactionInputZipBytes":928562409,"compactionOutputRawBytes":0,"compactionOutputZipBytes":903607394,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-db72c5678bc3c08cbdd44641fb71ab04-7624dffd25b8aa82-00"}}, response = 2024/11/23-15:19:16.463691 139957349447232 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-29064/att-00: olev 3, work 84.381 s, result 7.669 ms, install 1.591 ms, input{raw 3.030 GiB zip 885.546 MiB}