2024/11/22-21:23:35.572918 140025045571136 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00: fork to child time = 0.011858 sec 2024/11/22-21:23:35.573742 140025045571136 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00 2024/11/22-21:23:35.580453 140025045571136 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00 2024/11/22-21:23:35.581245 140025045571136 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-21:23:35.581515 140025045571136 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-21:23:35.634293 140025045571136 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-11402/att-00/MANIFEST-206347 2024/11/22-21:23:35.651247 140025045571136 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-11402/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 228514, last_sequence is 3572792807, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 228378 2024/11/22-21:23:35.651254 140025045571136 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-21:23:35.651341 140025045571136 [version_set.cc:5721] Creating manifest 228514 2024/11/22-21:23:35.682059 140025045571136 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-21:23:35.703043 140025045571136 [compaction_job.cc:2437] [default] [JOB 11402] Compacting 1@2 + 12@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/22-21:23:35.703061 140025045571136 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 8591 Base level 2, inputs: [228512(25MB)], [228450(35MB) 228451(35MB) 228452(36MB) 228453(35MB) 228471(35MB) 228472(38MB) 228473(35MB) 228474(35MB) 228475(36MB) 228476(35MB) 228477(33MB) 228478(35MB)] 2024/11/22-21:23:35.708920 140025045571136 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-21:23:35.708957 140025045571136 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-21:23:35.708977 140025045571136 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-21:23:35.708983 140025045571136 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-21:23:35.708996 140025045571136 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-21:23:36.158891 140025045571136 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228515.sst) /dev/shm/ToplingTemp/Topling-295832-xrhCVB 2024/11/22-21:23:36.734881 140025045571136 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228515.sst): first pass time = 0.58's, 233.039'MB/sec 2024/11/22-21:23:39.307510 140025045571136 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228515.sst): old prealloc_size = 147639500, real_size = 33166784 2024/11/22-21:23:39.308190 140025045571136 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228515.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928781 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.51 avg-zval = 33.56 2024/11/22-21:23:39.309075 140025045571136 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228515.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-21:23:41.115142 140025045571136 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228515.sst): second pass time = 2.57's, 42.077'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.81's, 20.619'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2681.850'MB/sec (index lex order gen) rebuild zvType time = 1.53's, 0.151'MB/sec write SST data time = 0.27's, 136.874'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32649.692'MB/sec zip my value time = 2.57's, unzip length = 0.108'GB zip my value throughput = 42.077'MB/sec zip pipeline throughput = 115.513'MB/sec entries = 928781 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.51 avg-zval = 33.56 usrkeys = 928781 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.51 avg-zval = 33.56 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928781 bytes = 3250752 } seq expand size = 6501467 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5138 value = 3.4719 dict = 1.00 all = 3.6011 } Zip/UnZip{ index = 0.0951 value = 0.2880 dict = 1.00 all = 0.2777 } ---------------------------- total value len = 0.114713 GB avg = 0.124 KB (by entry num) total key len = 0.026006 GB avg = 0.028 KB total ukey len = 0.018576 GB avg = 0.020 KB total ukey num = 0.000928781 Billion total entry num = 0.000928781 Billion write speed all = 0.003121684 MB/sec (with seq num) write speed all = 0.002956853 MB/sec (without seq num) 2024/11/22-21:23:41.155480 140025045571136 [compaction_job.cc:1948] [default] [JOB 11402] Generated table #228515: 928781 keys, 37270844 bytes, temperature: kWarm 2024/11/22-21:23:41.155597 140025045571136 EVENT_LOG_v1 {"time_micros": 1732281821155524, "cf_name": "default", "job": 11402, "event": "table_file_creation", "file_number": 228515, "file_size": 37270844, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038037, "largest_seqno": 3572776524, "table_properties": {"data_size": 31167600, "index_size": 1766784, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26005868, "raw_average_key_size": 28, "raw_value_size": 108211982, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928781, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732281816, "oldest_key_time": 0, "file_creation_time": 1732281816, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 228515, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-21:23:41.230501 140025045571136 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228516.sst) /dev/shm/ToplingTemp/Topling-295832-hUeUmC 2024/11/22-21:23:42.498463 140025045571136 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228516.sst): first pass time = 1.27's, 105.858'MB/sec 2024/11/22-21:23:44.981096 140025045571136 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228516.sst): old prealloc_size = 147639500, real_size = 32916160 2024/11/22-21:23:44.981661 140025045571136 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228516.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928260 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.59 avg-zval = 33.31 2024/11/22-21:23:44.982452 140025045571136 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228516.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-21:23:46.252602 140025045571136 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228516.sst): second pass time = 2.48's, 43.615'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.27's, 29.115'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3246.263'MB/sec (index lex order gen) rebuild zvType time = 0.89's, 0.262'MB/sec write SST data time = 0.38's, 96.212'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 28841.689'MB/sec zip my value time = 2.48's, unzip length = 0.108'GB zip my value throughput = 43.615'MB/sec zip pipeline throughput = 147.580'MB/sec entries = 928260 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.59 avg-zval = 33.31 usrkeys = 928260 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.59 avg-zval = 33.31 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928260 bytes = 3248928 } seq expand size = 6497820 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5125 value = 3.5005 dict = 1.00 all = 3.6257 } Zip/UnZip{ index = 0.0951 value = 0.2857 dict = 1.00 all = 0.2758 } ---------------------------- total value len = 0.229438 GB avg = 0.124 KB (by entry num) total key len = 0.051997 GB avg = 0.028 KB total ukey len = 0.037141 GB avg = 0.020 KB total ukey num = 0.001857041 Billion total entry num = 0.001857041 Billion write speed all = 0.006242575 MB/sec (with seq num) write speed all = 0.005913043 MB/sec (without seq num) 2024/11/22-21:23:46.288670 140025045571136 [compaction_job.cc:1948] [default] [JOB 11402] Generated table #228516: 928260 keys, 37018636 bytes, temperature: kWarm 2024/11/22-21:23:46.288752 140025045571136 EVENT_LOG_v1 {"time_micros": 1732281826288708, "cf_name": "default", "job": 11402, "event": "table_file_creation", "file_number": 228516, "file_size": 37018636, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3436178125, "largest_seqno": 3572776372, "table_properties": {"data_size": 30917872, "index_size": 1766016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25991280, "raw_average_key_size": 28, "raw_value_size": 108226568, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928260, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732281821, "oldest_key_time": 0, "file_creation_time": 1732281821, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 228516, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-21:23:46.359943 140025045571136 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228517.sst) /dev/shm/ToplingTemp/Topling-295832-VayhnE 2024/11/22-21:23:47.365193 140025045571136 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228517.sst): first pass time = 1.01's, 133.526'MB/sec 2024/11/22-21:23:49.520893 140025045571136 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228517.sst): old prealloc_size = 147639500, real_size = 33161072 2024/11/22-21:23:49.521429 140025045571136 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228517.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928573 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.54 avg-zval = 33.55 2024/11/22-21:23:49.522203 140025045571136 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228517.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-21:23:50.469447 140025045571136 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228517.sst): second pass time = 2.16's, 50.216'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.286'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3507.581'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.284'MB/sec write SST data time = 0.13's, 287.476'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31077.959'MB/sec zip my value time = 2.16's, unzip length = 0.108'GB zip my value throughput = 50.216'MB/sec zip pipeline throughput = 129.683'MB/sec entries = 928573 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.54 avg-zval = 33.55 usrkeys = 928573 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.54 avg-zval = 33.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928573 bytes = 3250016 } seq expand size = 6500011 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4445 value = 3.4740 dict = 1.00 all = 3.6017 } Zip/UnZip{ index = 0.0957 value = 0.2879 dict = 1.00 all = 0.2776 } ---------------------------- total value len = 0.344156 GB avg = 0.124 KB (by entry num) total key len = 0.077997 GB avg = 0.028 KB total ukey len = 0.055712 GB avg = 0.020 KB total ukey num = 0.002785614 Billion total entry num = 0.002785614 Billion write speed all = 0.009362993 MB/sec (with seq num) write speed all = 0.008868732 MB/sec (without seq num) 2024/11/22-21:23:50.503424 140025045571136 [compaction_job.cc:1948] [default] [JOB 11402] Generated table #228517: 928573 keys, 37264712 bytes, temperature: kWarm 2024/11/22-21:23:50.503619 140025045571136 EVENT_LOG_v1 {"time_micros": 1732281830503455, "cf_name": "default", "job": 11402, "event": "table_file_creation", "file_number": 228517, "file_size": 37264712, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038075, "largest_seqno": 3572776448, "table_properties": {"data_size": 31150624, "index_size": 1778112, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26000044, "raw_average_key_size": 28, "raw_value_size": 108217722, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928573, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732281826, "oldest_key_time": 0, "file_creation_time": 1732281826, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 228517, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-21:23:50.542199 140025045571136 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228518.sst) /dev/shm/ToplingTemp/Topling-295832-Gc1ZjB 2024/11/22-21:23:51.521058 140025045571136 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228518.sst): first pass time = 0.98's, 137.122'MB/sec 2024/11/22-21:23:53.245985 140025045571136 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228518.sst): old prealloc_size = 147639500, real_size = 33194960 2024/11/22-21:23:53.246594 140025045571136 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228518.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926165 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 116.92 avg-zval = 33.69 2024/11/22-21:23:53.247438 140025045571136 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228518.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-21:23:54.893936 140025045571136 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228518.sst): second pass time = 1.72's, 62.804'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.65's, 22.630'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3001.839'MB/sec (index lex order gen) rebuild zvType time = 1.37's, 0.169'MB/sec write SST data time = 0.27's, 136.256'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33581.666'MB/sec zip my value time = 1.72's, unzip length = 0.108'GB zip my value throughput = 62.804'MB/sec zip pipeline throughput = 62.804'MB/sec entries = 926165 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.92 avg-zval = 33.69 usrkeys = 926165 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.92 avg-zval = 33.69 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926165 bytes = 3241600 } seq expand size = 6483155 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5143 value = 3.4705 dict = 1.00 all = 3.5991 } Zip/UnZip{ index = 0.0951 value = 0.2881 dict = 1.00 all = 0.2778 } ---------------------------- total value len = 0.458924 GB avg = 0.124 KB (by entry num) total key len = 0.103930 GB avg = 0.028 KB total ukey len = 0.074236 GB avg = 0.020 KB total ukey num = 0.003711779 Billion total entry num = 0.003711779 Billion write speed all = 0.012482398 MB/sec (with seq num) write speed all = 0.011823870 MB/sec (without seq num) 2024/11/22-21:23:54.930707 140025045571136 [compaction_job.cc:1948] [default] [JOB 11402] Generated table #228518: 926165 keys, 37291554 bytes, temperature: kWarm 2024/11/22-21:23:54.930799 140025045571136 EVENT_LOG_v1 {"time_micros": 1732281834930744, "cf_name": "default", "job": 11402, "event": "table_file_creation", "file_number": 228518, "file_size": 37291554, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3436369950, "largest_seqno": 3572776623, "table_properties": {"data_size": 31201536, "index_size": 1761728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25932620, "raw_average_key_size": 28, "raw_value_size": 108285238, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926165, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732281830, "oldest_key_time": 0, "file_creation_time": 1732281830, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 228518, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-21:23:55.000629 140025045571136 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228519.sst) /dev/shm/ToplingTemp/Topling-295832-drPzJC 2024/11/22-21:23:56.033761 140025045571136 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228519.sst): first pass time = 1.03's, 129.920'MB/sec 2024/11/22-21:23:57.754031 140025045571136 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228519.sst): old prealloc_size = 147639500, real_size = 33644800 2024/11/22-21:23:57.754714 140025045571136 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228519.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926768 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 116.82 avg-zval = 34.15 2024/11/22-21:23:57.755555 140025045571136 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228519.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-21:23:59.420971 140025045571136 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228519.sst): second pass time = 1.72's, 62.968'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.67's, 22.642'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2667.300'MB/sec (index lex order gen) rebuild zvType time = 1.40's, 0.166'MB/sec write SST data time = 0.27's, 139.338'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 26142.791'MB/sec zip my value time = 1.72's, unzip length = 0.108'GB zip my value throughput = 62.968'MB/sec zip pipeline throughput = 84.641'MB/sec entries = 926768 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.82 avg-zval = 34.15 usrkeys = 926768 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.82 avg-zval = 34.15 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926768 bytes = 3243696 } seq expand size = 6487376 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5135 value = 3.4208 dict = 1.00 all = 3.5561 } Zip/UnZip{ index = 0.0951 value = 0.2923 dict = 1.00 all = 0.2812 } ---------------------------- total value len = 0.573680 GB avg = 0.124 KB (by entry num) total key len = 0.129879 GB avg = 0.028 KB total ukey len = 0.092771 GB avg = 0.020 KB total ukey num = 0.004638547 Billion total entry num = 0.004638547 Billion write speed all = 0.015601248 MB/sec (with seq num) write speed all = 0.014778379 MB/sec (without seq num) 2024/11/22-21:23:59.459104 140025045571136 [compaction_job.cc:1948] [default] [JOB 11402] Generated table #228519: 926768 keys, 37742802 bytes, temperature: kWarm 2024/11/22-21:23:59.459187 140025045571136 EVENT_LOG_v1 {"time_micros": 1732281839459141, "cf_name": "default", "job": 11402, "event": "table_file_creation", "file_number": 228519, "file_size": 37742802, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038136, "largest_seqno": 3572776112, "table_properties": {"data_size": 31649904, "index_size": 1763008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25949504, "raw_average_key_size": 28, "raw_value_size": 108268280, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926768, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732281834, "oldest_key_time": 0, "file_creation_time": 1732281834, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 228519, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-21:23:59.528969 140025045571136 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228520.sst) /dev/shm/ToplingTemp/Topling-295832-UEO4iB 2024/11/22-21:24:00.424289 140025045571136 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228520.sst): first pass time = 0.90's, 149.919'MB/sec 2024/11/22-21:24:02.001523 140025045571136 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228520.sst): old prealloc_size = 147639500, real_size = 36319328 2024/11/22-21:24:02.002184 140025045571136 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228520.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927076 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0343 GB avg-val = 116.78 avg-zval = 37.01 2024/11/22-21:24:02.002946 140025045571136 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228520.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-21:24:03.452702 140025045571136 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228520.sst): second pass time = 1.58's, 68.674'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.45's, 27.657'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2775.549'MB/sec (index lex order gen) rebuild zvType time = 1.08's, 0.214'MB/sec write SST data time = 0.37's, 108.957'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34935.364'MB/sec zip my value time = 1.58's, unzip length = 0.108'GB zip my value throughput = 68.674'MB/sec zip pipeline throughput = 68.674'MB/sec entries = 927076 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.78 avg-zval = 37.01 usrkeys = 927076 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.63 avg-zval = 37.01 TagRS{ kinds = RS_Key0_TagN bytes = 144960 } TagArray{ size = 776003 bytes = 2813024 } seq expand size = 5432021 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0343 GB dict = 1.08 MB all = 0.0401 GB } UnZip/Zip{ index = 10.4431 value = 3.1552 dict = 1.00 all = 3.3441 } Zip/UnZip{ index = 0.0958 value = 0.3169 dict = 1.00 all = 0.2990 } ---------------------------- total value len = 0.687371 GB avg = 0.124 KB (by entry num) total key len = 0.155837 GB avg = 0.028 KB total ukey len = 0.111312 GB avg = 0.020 KB total ukey num = 0.005565623 Billion total entry num = 0.005565623 Billion write speed all = 0.018696277 MB/sec (with seq num) write speed all = 0.017709034 MB/sec (without seq num) 2024/11/22-21:24:03.487955 140025045571136 [compaction_job.cc:1948] [default] [JOB 11402] Generated table #228520: 927076 keys, 40135502 bytes, temperature: kWarm 2024/11/22-21:24:03.488018 140025045571136 EVENT_LOG_v1 {"time_micros": 1732281843487982, "cf_name": "default", "job": 11402, "event": "table_file_creation", "file_number": 228520, "file_size": 40135502, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3572776879, "table_properties": {"data_size": 34311888, "index_size": 1775488, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25958128, "raw_average_key_size": 28, "raw_value_size": 108259668, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927076, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732281839, "oldest_key_time": 0, "file_creation_time": 1732281839, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 228520, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-21:24:03.525756 140025045571136 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228521.sst) /dev/shm/ToplingTemp/Topling-295832-YyUNWE 2024/11/22-21:24:04.279636 140025045571136 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228521.sst): first pass time = 0.75's, 178.049'MB/sec 2024/11/22-21:24:06.128793 140025045571136 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228521.sst): old prealloc_size = 147639500, real_size = 36454896 2024/11/22-21:24:06.129449 140025045571136 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228521.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926896 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.80 avg-zval = 37.18 2024/11/22-21:24:06.129512 140025045571136 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228521.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-21:24:07.267000 140025045571136 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228521.sst): second pass time = 1.85's, 58.576'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.14's, 32.782'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2786.844'MB/sec (index lex order gen) rebuild zvType time = 0.92's, 0.251'MB/sec write SST data time = 0.22's, 173.181'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32723.515'MB/sec zip my value time = 1.85's, unzip length = 0.108'GB zip my value throughput = 58.576'MB/sec zip pipeline throughput = 58.576'MB/sec entries = 926896 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.80 avg-zval = 37.18 usrkeys = 926896 avg-key = 20.00 avg-zkey = 1.90 avg-val = 116.80 avg-zval = 37.18 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 10 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5134 value = 3.1418 dict = 1.00 all = 3.5971 } Zip/UnZip{ index = 0.0951 value = 0.3183 dict = 1.00 all = 0.2780 } ---------------------------- total value len = 0.795636 GB avg = 0.123 KB (by entry num) total key len = 0.181791 GB avg = 0.028 KB total ukey len = 0.129850 GB avg = 0.020 KB total ukey num = 0.006492519 Billion total entry num = 0.006492519 Billion write speed all = 0.021670426 MB/sec (with seq num) write speed all = 0.020518866 MB/sec (without seq num) 2024/11/22-21:24:07.299827 140025045571136 [compaction_job.cc:1948] [default] [JOB 11402] Generated table #228521: 926896 keys, 37312646 bytes, temperature: kWarm 2024/11/22-21:24:07.299894 140025045571136 EVENT_LOG_v1 {"time_micros": 1732281847299863, "cf_name": "default", "job": 11402, "event": "table_file_creation", "file_number": 228521, "file_size": 37312646, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34459744, "index_size": 1763264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25953088, "raw_average_key_size": 28, "raw_value_size": 108264756, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926896, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732281843, "oldest_key_time": 0, "file_creation_time": 1732281843, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 228521, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-21:24:07.338140 140025045571136 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228522.sst) /dev/shm/ToplingTemp/Topling-295832-EPTKaB 2024/11/22-21:24:08.143018 140025045571136 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228522.sst): first pass time = 0.80's, 166.764'MB/sec 2024/11/22-21:24:10.020218 140025045571136 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228522.sst): old prealloc_size = 147639500, real_size = 36463824 2024/11/22-21:24:10.020879 140025045571136 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228522.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927074 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.78 avg-zval = 37.17 2024/11/22-21:24:10.020945 140025045571136 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228522.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-21:24:11.118769 140025045571136 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228522.sst): second pass time = 1.88's, 57.727'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.10's, 33.974'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2774.963'MB/sec (index lex order gen) rebuild zvType time = 0.93's, 0.249'MB/sec write SST data time = 0.17's, 223.229'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35109.125'MB/sec zip my value time = 1.88's, unzip length = 0.108'GB zip my value throughput = 57.727'MB/sec zip pipeline throughput = 57.727'MB/sec entries = 927074 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.78 avg-zval = 37.17 usrkeys = 927074 avg-key = 20.00 avg-zkey = 1.92 avg-val = 116.78 avg-zval = 37.17 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 10 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4438 value = 3.1419 dict = 1.00 all = 3.5962 } Zip/UnZip{ index = 0.0958 value = 0.3183 dict = 1.00 all = 0.2781 } ---------------------------- total value len = 0.903896 GB avg = 0.122 KB (by entry num) total key len = 0.207749 GB avg = 0.028 KB total ukey len = 0.148392 GB avg = 0.020 KB total ukey num = 0.007419593 Billion total entry num = 0.007419593 Billion write speed all = 0.024644049 MB/sec (with seq num) write speed all = 0.023328169 MB/sec (without seq num) 2024/11/22-21:24:11.155926 140025045571136 [compaction_job.cc:1948] [default] [JOB 11402] Generated table #228522: 927074 keys, 37321664 bytes, temperature: kWarm 2024/11/22-21:24:11.155988 140025045571136 EVENT_LOG_v1 {"time_micros": 1732281851155957, "cf_name": "default", "job": 11402, "event": "table_file_creation", "file_number": 228522, "file_size": 37321664, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34456512, "index_size": 1775360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25958072, "raw_average_key_size": 28, "raw_value_size": 108259656, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927074, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732281847, "oldest_key_time": 0, "file_creation_time": 1732281847, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 228522, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-21:24:11.191121 140025045571136 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228523.sst) /dev/shm/ToplingTemp/Topling-295832-wGjjtB 2024/11/22-21:24:12.006335 140025045571136 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228523.sst): first pass time = 0.82's, 164.649'MB/sec 2024/11/22-21:24:13.486582 140025045571136 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228523.sst): old prealloc_size = 147639500, real_size = 34747648 2024/11/22-21:24:13.487169 140025045571136 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228523.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922249 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1084 GB zip-val = 0.0328 GB avg-val = 117.53 avg-zval = 35.54 2024/11/22-21:24:13.487871 140025045571136 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228523.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-21:24:14.368493 140025045571136 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228523.sst): second pass time = 1.48's, 73.263'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 43.623'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3155.263'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.308'MB/sec write SST data time = 0.13's, 288.035'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34003.892'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.263'MB/sec zip pipeline throughput = 73.263'MB/sec entries = 922249 avg-key = 28.00 avg-zkey = 1.89 avg-val = 117.53 avg-zval = 35.54 usrkeys = 922249 avg-key = 20.00 avg-zkey = 1.89 avg-val = 123.22 avg-zval = 35.54 TagRS{ kinds = RS_Key0_TagN bytes = 144200 } TagArray{ size = 749521 bytes = 2717024 } seq expand size = 5246647 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0328 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.5794 value = 3.3074 dict = 1.00 all = 3.4888 } Zip/UnZip{ index = 0.0945 value = 0.3024 dict = 1.00 all = 0.2866 } ---------------------------- total value len = 1.017537 GB avg = 0.122 KB (by entry num) total key len = 0.233572 GB avg = 0.028 KB total ukey len = 0.166837 GB avg = 0.020 KB total ukey num = 0.008341842 Billion total entry num = 0.008341842 Billion write speed all = 0.027733838 MB/sec (with seq num) write speed all = 0.026254502 MB/sec (without seq num) 2024/11/22-21:24:14.403103 140025045571136 [compaction_job.cc:1948] [default] [JOB 11402] Generated table #228523: 922249 keys, 38471156 bytes, temperature: kWarm 2024/11/22-21:24:14.403166 140025045571136 EVENT_LOG_v1 {"time_micros": 1732281854403135, "cf_name": "default", "job": 11402, "event": "table_file_creation", "file_number": 228523, "file_size": 38471156, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3572776601, "table_properties": {"data_size": 32773424, "index_size": 1743488, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25822972, "raw_average_key_size": 28, "raw_value_size": 108394766, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922249, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732281851, "oldest_key_time": 0, "file_creation_time": 1732281851, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 228523, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-21:24:14.442584 140025045571136 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228524.sst) /dev/shm/ToplingTemp/Topling-295832-LXAAcF 2024/11/22-21:24:15.509718 140025045571136 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228524.sst): first pass time = 1.07's, 125.780'MB/sec 2024/11/22-21:24:17.059909 140025045571136 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228524.sst): old prealloc_size = 147639500, real_size = 33128016 2024/11/22-21:24:17.060556 140025045571136 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228524.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928256 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.59 avg-zval = 33.54 2024/11/22-21:24:17.063864 140025045571136 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228524.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-21:24:18.190622 140025045571136 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228524.sst): second pass time = 1.55's, 69.848'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.13's, 32.824'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2757.242'MB/sec (index lex order gen) rebuild zvType time = 0.97's, 0.239'MB/sec write SST data time = 0.16's, 233.208'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33821.884'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 69.848'MB/sec zip pipeline throughput = 69.848'MB/sec entries = 928256 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.59 avg-zval = 33.54 usrkeys = 928256 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.59 avg-zval = 33.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928256 bytes = 3132880 } seq expand size = 6497792 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5124 value = 3.4766 dict = 1.00 all = 3.6163 } Zip/UnZip{ index = 0.0951 value = 0.2876 dict = 1.00 all = 0.2765 } ---------------------------- total value len = 1.132261 GB avg = 0.122 KB (by entry num) total key len = 0.259563 GB avg = 0.028 KB total ukey len = 0.185402 GB avg = 0.020 KB total ukey num = 0.009270098 Billion total entry num = 0.009270098 Billion write speed all = 0.030850522 MB/sec (with seq num) write speed all = 0.029206709 MB/sec (without seq num) 2024/11/22-21:24:18.224787 140025045571136 [compaction_job.cc:1948] [default] [JOB 11402] Generated table #228524: 928256 keys, 37114336 bytes, temperature: kWarm 2024/11/22-21:24:18.224853 140025045571136 EVENT_LOG_v1 {"time_micros": 1732281858224819, "cf_name": "default", "job": 11402, "event": "table_file_creation", "file_number": 228524, "file_size": 37114336, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3479654129, "largest_seqno": 3572775628, "table_properties": {"data_size": 31129728, "index_size": 1766016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25991168, "raw_average_key_size": 28, "raw_value_size": 108226560, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928256, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732281854, "oldest_key_time": 0, "file_creation_time": 1732281854, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 228524, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-21:24:18.263180 140025045571136 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228525.sst) /dev/shm/ToplingTemp/Topling-295832-rVG0XD 2024/11/22-21:24:19.276486 140025045571136 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228525.sst): first pass time = 1.01's, 132.461'MB/sec 2024/11/22-21:24:20.544350 140025045571136 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228525.sst): old prealloc_size = 147639500, real_size = 32022592 2024/11/22-21:24:20.544945 140025045571136 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228525.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920177 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1085 GB zip-val = 0.0300 GB avg-val = 117.86 avg-zval = 32.64 2024/11/22-21:24:20.547785 140025045571136 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228525.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-21:24:21.432041 140025045571136 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228525.sst): second pass time = 1.27's, 85.592'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 40.801'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3073.926'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.305'MB/sec write SST data time = 0.13's, 273.791'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 31192.935'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 85.592'MB/sec zip pipeline throughput = 85.592'MB/sec entries = 920177 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.86 avg-zval = 32.64 usrkeys = 920177 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.86 avg-zval = 32.64 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 920177 bytes = 3335664 } seq expand size = 6441239 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.4425 value = 3.6115 dict = 1.00 all = 3.7058 } Zip/UnZip{ index = 0.0958 value = 0.2769 dict = 1.00 all = 0.2698 } ---------------------------- total value len = 1.247156 GB avg = 0.122 KB (by entry num) total key len = 0.285328 GB avg = 0.028 KB total ukey len = 0.203806 GB avg = 0.020 KB total ukey num = 0.010190275 Billion total entry num = 0.010190275 Billion write speed all = 0.033965863 MB/sec (with seq num) write speed all = 0.032159010 MB/sec (without seq num) 2024/11/22-21:24:21.464679 140025045571136 [compaction_job.cc:1948] [default] [JOB 11402] Generated table #228525: 920177 keys, 36218392 bytes, temperature: kWarm 2024/11/22-21:24:21.464744 140025045571136 EVENT_LOG_v1 {"time_micros": 1732281861464710, "cf_name": "default", "job": 11402, "event": "table_file_creation", "file_number": 228525, "file_size": 36218392, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701386, "largest_seqno": 3572776400, "table_properties": {"data_size": 30030000, "index_size": 1762368, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25764956, "raw_average_key_size": 28, "raw_value_size": 108452806, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920177, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732281858, "oldest_key_time": 0, "file_creation_time": 1732281858, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 228525, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-21:24:21.515733 140025045571136 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228526.sst) /dev/shm/ToplingTemp/Topling-295832-9r23pE 2024/11/22-21:24:22.426818 140025045571136 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228526.sst): first pass time = 0.91's, 147.324'MB/sec 2024/11/22-21:24:23.781119 140025045571136 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228526.sst): old prealloc_size = 147639500, real_size = 33736496 2024/11/22-21:24:23.781661 140025045571136 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228526.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924214 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0317 GB avg-val = 117.22 avg-zval = 34.35 2024/11/22-21:24:23.784532 140025045571136 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228526.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-21:24:24.727004 140025045571136 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228526.sst): second pass time = 1.35's, 80.035'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.993'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3386.187'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.284'MB/sec write SST data time = 0.13's, 286.424'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 29877.879'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.035'MB/sec zip pipeline throughput = 80.035'MB/sec entries = 924214 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.22 avg-zval = 34.35 usrkeys = 924214 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.22 avg-zval = 34.35 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924214 bytes = 3234768 } seq expand size = 6469498 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.5112 value = 3.4126 dict = 1.00 all = 3.5481 } Zip/UnZip{ index = 0.0951 value = 0.2930 dict = 1.00 all = 0.2818 } ---------------------------- total value len = 1.361965 GB avg = 0.123 KB (by entry num) total key len = 0.311206 GB avg = 0.028 KB total ukey len = 0.222290 GB avg = 0.020 KB total ukey num = 0.011114489 Billion total entry num = 0.011114489 Billion write speed all = 0.037081340 MB/sec (with seq num) write speed all = 0.035110757 MB/sec (without seq num) 2024/11/22-21:24:24.763711 140025045571136 [compaction_job.cc:1948] [default] [JOB 11402] Generated table #228526: 924214 keys, 37827942 bytes, temperature: kWarm 2024/11/22-21:24:24.763775 140025045571136 EVENT_LOG_v1 {"time_micros": 1732281864763742, "cf_name": "default", "job": 11402, "event": "table_file_creation", "file_number": 228526, "file_size": 37827942, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3378289821, "largest_seqno": 3572370988, "table_properties": {"data_size": 31746720, "index_size": 1758528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25877992, "raw_average_key_size": 28, "raw_value_size": 108339820, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924214, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732281861, "oldest_key_time": 0, "file_creation_time": 1732281861, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 228526, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-21:24:24.800574 140025045571136 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228527.sst) /dev/shm/ToplingTemp/Topling-295832-qJDz8E 2024/11/22-21:24:24.879496 140025045571136 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228527.sst): first pass time = 0.08's, 229.911'MB/sec 2024/11/22-21:24:25.037257 140025045571136 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228527.sst): old prealloc_size = 147639500, real_size = 5397792 2024/11/22-21:24:25.037381 140025045571136 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228527.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 124215 prefix = 4 raw-key = 0.0020 GB zip-key = 0.0002 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0147 GB zip-val = 0.0051 GB avg-val = 118.00 avg-zval = 41.28 2024/11/22-21:24:25.037527 140025045571136 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228527.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.018 GB 2024/11/22-21:24:25.153263 140025045571136 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00/228527.sst): second pass time = 0.16's, 93.253'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.12's, 50.952'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2343.473'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.316'MB/sec write SST data time = 0.02's, 335.277'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.154'MB, throughput = 5300.169'MB/sec zip my value time = 0.16's, unzip length = 0.015'GB zip my value throughput = 93.253'MB/sec zip pipeline throughput = 93.253'MB/sec entries = 124215 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 41.28 usrkeys = 124215 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 41.28 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 124215 bytes = 388192 } seq expand size = 869505 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0025 GB value = 0.0147 GB dict = 0.15 MB all = 0.0181 GB } __ZipSize{ index = 0.0002 GB value = 0.0051 GB dict = 0.15 MB all = 0.0059 GB } UnZip/Zip{ index = 10.3789 value = 2.8588 dict = 1.00 all = 3.0686 } Zip/UnZip{ index = 0.0963 value = 0.3498 dict = 1.00 all = 0.3259 } ---------------------------- total value len = 1.377492 GB avg = 0.123 KB (by entry num) total key len = 0.314684 GB avg = 0.028 KB total ukey len = 0.224774 GB avg = 0.020 KB total ukey num = 0.011238704 Billion total entry num = 0.011238704 Billion write speed all = 0.037502178 MB/sec (with seq num) write speed all = 0.035509591 MB/sec (without seq num) 2024/11/22-21:24:25.163464 140025045571136 [compaction_job.cc:1948] [default] [JOB 11402] Generated table #228527: 124215 keys, 5909955 bytes, temperature: kWarm 2024/11/22-21:24:25.163522 140025045571136 EVENT_LOG_v1 {"time_micros": 1732281865163491, "cf_name": "default", "job": 11402, "event": "table_file_creation", "file_number": 228527, "file_size": 5909955, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3487609057, "largest_seqno": 3512876623, "table_properties": {"data_size": 5127184, "index_size": 239360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3478020, "raw_average_key_size": 28, "raw_value_size": 14657370, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 124215, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732281864, "oldest_key_time": 0, "file_creation_time": 1732281864, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 228527, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-21:24:25.174625 140025045571136 [compaction_job.cc:703] job-11402: subcompact[0], size: 0.456900 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11402/att-00 [228515,228516,228517,228518,228519,228520,228521,228522,228523,228524,228525,228526,228527] 2024/11/22-21:24:25.276396 140025045571136 (Original Log Time 2024/11/22-21:23:35.703117) EVENT_LOG_v1 {"time_micros": 1732281815703073, "job": 11402, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [228512], "files_L3": [228450, 228451, 228452, 228453, 228471, 228472, 228473, 228474, 228475, 228476, 228477, 228478], "score": 0.998675, "input_data_size": 477285348, "oldest_snapshot_seqno": 3572792746} 2024/11/22-21:24:25.276399 140025045571136 (Original Log Time 2024/11/22-21:24:25.275692) [compaction_job.cc:2022] [default] [JOB 11402] Compacted 1@2 + 12@3 files to L3 => 456900141 bytes 2024/11/22-21:24:25.276401 140025045571136 (Original Log Time 2024/11/22-21:24:25.276321) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 13 0 0 0] max score 0.02, MB/sec: 9.6 rd, 9.2 wr, level 3, files in(1, 12) out(13 +0 blob) MB in(25.0, 430.2 +0.0 blob) out(435.7 +0.0 blob), read-write-amplify(35.6) write-amplify(17.4) OK, records in: 11328627, records dropped: 89923 output_compression: Snappy 2024/11/22-21:24:25.276407 140025045571136 (Original Log Time 2024/11/22-21:24:25.276377) EVENT_LOG_v1 {"time_micros": 1732281865276346, "cf": "default", "job": 11402, "event": "compaction_finished", "compaction_time_micros": 49471515, "compaction_time_cpu_micros": 21134348, "output_level": 3, "num_output_files": 13, "total_output_size": 456900141, "num_input_records": 11328627, "num_output_records": 11238704, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 381352068, "file_range_sync_nanos": 0, "file_fsync_nanos": 12568, "file_prepare_write_nanos": 36077, "lsm_state": [0, 0, 0, 13, 0, 0, 0]} 2024/11/22-21:24:25.411741 140025045571136 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732281815,"executesMs":49696,"compactionJobId":11402,"attempt":0,"compactionInputRawBytes":1641517970,"compactionInputZipBytes":477285348,"compactionOutputRawBytes":0,"compactionOutputZipBytes":456900141,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-4ad1836a487cff2663c549dd18c96d83-5492be9b0bc189b8-00"}}, response = 2024/11/22-21:24:25.411831 140025045571136 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-11402/att-00: olev 3, work 49.696 s, result 6.301 ms, install 0.733 ms, input{raw 1.529 GiB zip 455.175 MiB}