2024/11/23-13:40:30.174345 139957173200448 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00: fork to child time = 0.008018 sec 2024/11/23-13:40:30.174930 139957173200448 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00 2024/11/23-13:40:30.181642 139957173200448 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00 2024/11/23-13:40:30.182498 139957173200448 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-13:40:30.182739 139957173200448 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-13:40:30.238515 139957173200448 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-26959/att-00/MANIFEST-206347 2024/11/23-13:40:30.255151 139957173200448 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-26959/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 261550, last_sequence is 4000531596, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 261401 2024/11/23-13:40:30.255157 139957173200448 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-13:40:30.255261 139957173200448 [version_set.cc:5721] Creating manifest 261550 2024/11/23-13:40:30.297041 139957173200448 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-13:40:30.322935 139957173200448 [compaction_job.cc:2437] [default] [JOB 26959] Compacting 1@2 + 13@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-13:40:30.322953 139957173200448 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 20192 Base level 2, inputs: [261547(25MB)], [261492(35MB) 261493(35MB) 261494(36MB) 261505(35MB) 261506(35MB) 261507(35MB) 261508(33MB) 261509(35MB) 261510(18MB) 261467(33MB) 261468(34MB) 261469(35MB) 261470(3723KB)] 2024/11/23-13:40:30.329664 139957173200448 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-13:40:30.329691 139957173200448 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-13:40:30.329708 139957173200448 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-13:40:30.329713 139957173200448 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-13:40:30.329724 139957173200448 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-13:40:30.917201 139957173200448 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261551.sst) /dev/shm/ToplingTemp/Topling-62180-uJ2T8X 2024/11/23-13:40:31.469130 139957173200448 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261551.sst): first pass time = 0.55's, 243.194'MB/sec 2024/11/23-13:40:32.979301 139957173200448 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261551.sst): old prealloc_size = 147639500, real_size = 32722352 2024/11/23-13:40:32.980059 139957173200448 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261551.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931149 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0307 GB avg-val = 116.14 avg-zval = 32.98 2024/11/23-13:40:32.980995 139957173200448 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261551.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:40:34.308221 139957173200448 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261551.sst): second pass time = 1.51's, 71.657'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.33's, 27.805'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2432.052'MB/sec (index lex order gen) rebuild zvType time = 1.15's, 0.203'MB/sec write SST data time = 0.18's, 205.850'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 41768.820'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.657'MB/sec zip pipeline throughput = 71.657'MB/sec entries = 931149 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.14 avg-zval = 32.98 usrkeys = 931149 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.14 avg-zval = 32.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931149 bytes = 3375424 } seq expand size = 6518043 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4449 value = 3.5219 dict = 1.00 all = 3.6325 } Zip/UnZip{ index = 0.0957 value = 0.2839 dict = 1.00 all = 0.2753 } ---------------------------- total value len = 8.963283 GB avg = 0.168 KB (by entry num) total key len = 1.466893 GB avg = 0.028 KB total ukey len = 1.016501 GB avg = 0.015 KB total ukey num = 0.066481584 Billion total entry num = 0.053289313 Billion write speed all = 0.609164801 MB/sec (with seq num) write speed all = 0.584266298 MB/sec (without seq num) 2024/11/23-13:40:34.355502 139957173200448 [compaction_job.cc:1948] [default] [JOB 26959] Generated table #261551: 931149 keys, 36949502 bytes, temperature: kWarm 2024/11/23-13:40:34.355630 139957173200448 EVENT_LOG_v1 {"time_micros": 1732340434355546, "cf_name": "default", "job": 26959, "event": "table_file_creation", "file_number": 261551, "file_size": 36949502, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3610601794, "largest_seqno": 4000518481, "table_properties": {"data_size": 30706400, "index_size": 1782976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26072172, "raw_average_key_size": 28, "raw_value_size": 108145634, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931149, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340430, "oldest_key_time": 0, "file_creation_time": 1732340430, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261551, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:40:34.408768 139957173200448 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261552.sst) /dev/shm/ToplingTemp/Topling-62180-KLG06V 2024/11/23-13:40:35.634761 139957173200448 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261552.sst): first pass time = 1.23's, 109.482'MB/sec 2024/11/23-13:40:37.330400 139957173200448 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261552.sst): old prealloc_size = 147639500, real_size = 32344272 2024/11/23-13:40:37.331129 139957173200448 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261552.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931300 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0303 GB avg-val = 116.12 avg-zval = 32.58 2024/11/23-13:40:37.331999 139957173200448 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261552.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:40:39.105928 139957173200448 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261552.sst): second pass time = 1.69's, 63.808'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.78's, 20.598'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2505.628'MB/sec (index lex order gen) rebuild zvType time = 1.57's, 0.148'MB/sec write SST data time = 0.20's, 179.570'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33013.226'MB/sec zip my value time = 1.69's, unzip length = 0.108'GB zip my value throughput = 63.808'MB/sec zip pipeline throughput = 63.808'MB/sec entries = 931300 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.12 avg-zval = 32.58 usrkeys = 931300 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.12 avg-zval = 32.58 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931300 bytes = 3375984 } seq expand size = 6519100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5122 value = 3.5644 dict = 1.00 all = 3.6700 } Zip/UnZip{ index = 0.0951 value = 0.2806 dict = 1.00 all = 0.2725 } ---------------------------- total value len = 9.077944 GB avg = 0.167 KB (by entry num) total key len = 1.492970 GB avg = 0.028 KB total ukey len = 1.035127 GB avg = 0.015 KB total ukey num = 0.067412884 Billion total entry num = 0.054220613 Billion write speed all = 0.617211463 MB/sec (with seq num) write speed all = 0.591884923 MB/sec (without seq num) 2024/11/23-13:40:39.158764 139957173200448 [compaction_job.cc:1948] [default] [JOB 26959] Generated table #261552: 931300 keys, 36571544 bytes, temperature: kWarm 2024/11/23-13:40:39.158825 139957173200448 EVENT_LOG_v1 {"time_micros": 1732340439158792, "cf_name": "default", "job": 26959, "event": "table_file_creation", "file_number": 261552, "file_size": 36571544, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 4000518653, "table_properties": {"data_size": 30339392, "index_size": 1771840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26076400, "raw_average_key_size": 28, "raw_value_size": 108141416, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340434, "oldest_key_time": 0, "file_creation_time": 1732340434, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261552, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:40:39.194956 139957173200448 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261553.sst) /dev/shm/ToplingTemp/Topling-62180-CCB9gV 2024/11/23-13:40:40.263438 139957173200448 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261553.sst): first pass time = 1.07's, 125.622'MB/sec 2024/11/23-13:40:41.663621 139957173200448 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261553.sst): old prealloc_size = 147639500, real_size = 33516896 2024/11/23-13:40:41.664309 139957173200448 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261553.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925741 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 116.98 avg-zval = 34.05 2024/11/23-13:40:41.665198 139957173200448 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261553.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:40:43.278923 139957173200448 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261553.sst): second pass time = 1.40's, 77.437'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.62's, 23.428'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2642.138'MB/sec (index lex order gen) rebuild zvType time = 1.36's, 0.170'MB/sec write SST data time = 0.26's, 147.895'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 28381.991'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.437'MB/sec zip pipeline throughput = 78.337'MB/sec entries = 925741 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.98 avg-zval = 34.05 usrkeys = 925741 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.98 avg-zval = 34.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925741 bytes = 3471536 } seq expand size = 6480187 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.5152 value = 3.4353 dict = 1.00 all = 3.5467 } Zip/UnZip{ index = 0.0951 value = 0.2911 dict = 1.00 all = 0.2820 } ---------------------------- total value len = 9.192721 GB avg = 0.167 KB (by entry num) total key len = 1.518890 GB avg = 0.028 KB total ukey len = 1.053642 GB avg = 0.015 KB total ukey num = 0.068338625 Billion total entry num = 0.055146354 Billion write speed all = 0.625274150 MB/sec (with seq num) write speed all = 0.599521469 MB/sec (without seq num) 2024/11/23-13:40:43.328921 139957173200448 [compaction_job.cc:1948] [default] [JOB 26959] Generated table #261553: 925741 keys, 37843242 bytes, temperature: kWarm 2024/11/23-13:40:43.329003 139957173200448 EVENT_LOG_v1 {"time_micros": 1732340443328954, "cf_name": "default", "job": 26959, "event": "table_file_creation", "file_number": 261553, "file_size": 37843242, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 4000518582, "table_properties": {"data_size": 31524496, "index_size": 1760768, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25920748, "raw_average_key_size": 28, "raw_value_size": 108297078, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925741, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340439, "oldest_key_time": 0, "file_creation_time": 1732340439, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261553, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:40:43.368222 139957173200448 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261554.sst) /dev/shm/ToplingTemp/Topling-62180-PXhTMX 2024/11/23-13:40:44.397383 139957173200448 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261554.sst): first pass time = 1.03's, 130.421'MB/sec 2024/11/23-13:40:46.419338 139957173200448 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261554.sst): old prealloc_size = 147639500, real_size = 33605344 2024/11/23-13:40:46.420089 139957173200448 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261554.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933032 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0316 GB avg-val = 115.85 avg-zval = 33.85 2024/11/23-13:40:46.420919 139957173200448 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261554.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:40:47.922726 139957173200448 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261554.sst): second pass time = 2.02's, 53.481'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.50's, 24.890'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2442.288'MB/sec (index lex order gen) rebuild zvType time = 1.23's, 0.189'MB/sec write SST data time = 0.27's, 138.523'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35533.169'MB/sec zip my value time = 2.02's, unzip length = 0.108'GB zip my value throughput = 53.481'MB/sec zip pipeline throughput = 118.607'MB/sec entries = 933032 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.85 avg-zval = 33.85 usrkeys = 933032 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.48 avg-zval = 33.85 TagRS{ kinds = RS_Key0_TagN bytes = 145880 } TagArray{ size = 750035 bytes = 2812640 } seq expand size = 5250245 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4446 value = 3.4223 dict = 1.00 all = 3.5869 } Zip/UnZip{ index = 0.0957 value = 0.2922 dict = 1.00 all = 0.2788 } ---------------------------- total value len = 9.306064 GB avg = 0.166 KB (by entry num) total key len = 1.545015 GB avg = 0.028 KB total ukey len = 1.072302 GB avg = 0.015 KB total ukey num = 0.069271657 Billion total entry num = 0.056079386 Billion write speed all = 0.633243730 MB/sec (with seq num) write speed all = 0.607062432 MB/sec (without seq num) 2024/11/23-13:40:47.974184 139957173200448 [compaction_job.cc:1948] [default] [JOB 26959] Generated table #261554: 933032 keys, 37418558 bytes, temperature: kWarm 2024/11/23-13:40:47.974271 139957173200448 EVENT_LOG_v1 {"time_micros": 1732340447974223, "cf_name": "default", "job": 26959, "event": "table_file_creation", "file_number": 261554, "file_size": 37418558, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4000518262, "table_properties": {"data_size": 31585296, "index_size": 1786624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26124896, "raw_average_key_size": 28, "raw_value_size": 108092896, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933032, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340443, "oldest_key_time": 0, "file_creation_time": 1732340443, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261554, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:40:48.028704 139957173200448 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261555.sst) /dev/shm/ToplingTemp/Topling-62180-711nbY 2024/11/23-13:40:48.979822 139957173200448 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261555.sst): first pass time = 0.95's, 141.125'MB/sec 2024/11/23-13:40:51.303336 139957173200448 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261555.sst): old prealloc_size = 147639500, real_size = 32982928 2024/11/23-13:40:51.303890 139957173200448 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261555.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930080 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.31 avg-zval = 33.31 2024/11/23-13:40:51.304740 139957173200448 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261555.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:40:52.380303 139957173200448 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261555.sst): second pass time = 2.32's, 46.576'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.08's, 34.549'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3392.332'MB/sec (index lex order gen) rebuild zvType time = 0.84's, 0.277'MB/sec write SST data time = 0.24's, 157.795'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 36323.953'MB/sec zip my value time = 2.32's, unzip length = 0.108'GB zip my value throughput = 46.576'MB/sec zip pipeline throughput = 122.285'MB/sec entries = 930080 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.31 avg-zval = 33.31 usrkeys = 930080 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.31 avg-zval = 33.31 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930080 bytes = 3371552 } seq expand size = 6510560 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5144 value = 3.4917 dict = 1.00 all = 3.6073 } Zip/UnZip{ index = 0.0951 value = 0.2864 dict = 1.00 all = 0.2772 } ---------------------------- total value len = 9.420751 GB avg = 0.165 KB (by entry num) total key len = 1.571058 GB avg = 0.028 KB total ukey len = 1.090904 GB avg = 0.016 KB total ukey num = 0.070201737 Billion total entry num = 0.057009466 Billion write speed all = 0.641289488 MB/sec (with seq num) write speed all = 0.614680893 MB/sec (without seq num) 2024/11/23-13:40:52.431959 139957173200448 [compaction_job.cc:1948] [default] [JOB 26959] Generated table #261555: 930080 keys, 37206872 bytes, temperature: kWarm 2024/11/23-13:40:52.432063 139957173200448 EVENT_LOG_v1 {"time_micros": 1732340452432014, "cf_name": "default", "job": 26959, "event": "table_file_creation", "file_number": 261555, "file_size": 37206872, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4000518242, "table_properties": {"data_size": 30981056, "index_size": 1769152, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26042240, "raw_average_key_size": 28, "raw_value_size": 108175556, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930080, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340448, "oldest_key_time": 0, "file_creation_time": 1732340448, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261555, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:40:52.484485 139957173200448 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261556.sst) /dev/shm/ToplingTemp/Topling-62180-JQTILX 2024/11/23-13:40:53.396712 139957173200448 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261556.sst): first pass time = 0.91's, 147.143'MB/sec 2024/11/23-13:40:55.250112 139957173200448 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261556.sst): old prealloc_size = 147639500, real_size = 33287648 2024/11/23-13:40:55.250708 139957173200448 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261556.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 935096 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1080 GB zip-val = 0.0313 GB avg-val = 115.53 avg-zval = 33.42 2024/11/23-13:40:55.252677 139957173200448 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261556.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:40:56.331535 139957173200448 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261556.sst): second pass time = 1.85's, 58.315'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.08's, 34.703'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3118.429'MB/sec (index lex order gen) rebuild zvType time = 0.85's, 0.276'MB/sec write SST data time = 0.23's, 161.517'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32047.211'MB/sec zip my value time = 1.85's, unzip length = 0.108'GB zip my value throughput = 58.315'MB/sec zip pipeline throughput = 119.066'MB/sec entries = 935096 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.53 avg-zval = 33.42 usrkeys = 935096 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.53 avg-zval = 33.42 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 935096 bytes = 3389744 } seq expand size = 6545672 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.3770 value = 3.4570 dict = 1.00 all = 3.5765 } Zip/UnZip{ index = 0.0964 value = 0.2893 dict = 1.00 all = 0.2796 } ---------------------------- total value len = 9.535331 GB avg = 0.165 KB (by entry num) total key len = 1.597240 GB avg = 0.028 KB total ukey len = 1.109606 GB avg = 0.016 KB total ukey num = 0.071136833 Billion total entry num = 0.057944562 Billion write speed all = 0.649352290 MB/sec (with seq num) write speed all = 0.622313482 MB/sec (without seq num) 2024/11/23-13:40:56.382171 139957173200448 [compaction_job.cc:1948] [default] [JOB 26959] Generated table #261556: 935096 keys, 37527428 bytes, temperature: kWarm 2024/11/23-13:40:56.382262 139957173200448 EVENT_LOG_v1 {"time_micros": 1732340456382214, "cf_name": "default", "job": 26959, "event": "table_file_creation", "file_number": 261556, "file_size": 37527428, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4000518400, "table_properties": {"data_size": 31251472, "index_size": 1802240, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26182688, "raw_average_key_size": 28, "raw_value_size": 108035148, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 935096, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340452, "oldest_key_time": 0, "file_creation_time": 1732340452, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261556, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:40:56.454674 139957173200448 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261557.sst) /dev/shm/ToplingTemp/Topling-62180-Pg8aWZ 2024/11/23-13:40:57.487822 139957173200448 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261557.sst): first pass time = 1.03's, 129.919'MB/sec 2024/11/23-13:40:59.037396 139957173200448 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261557.sst): old prealloc_size = 147639500, real_size = 32432864 2024/11/23-13:40:59.037943 139957173200448 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261557.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926095 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0304 GB avg-val = 116.93 avg-zval = 32.87 2024/11/23-13:40:59.038747 139957173200448 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261557.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:41:00.080262 139957173200448 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261557.sst): second pass time = 1.55's, 69.945'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.04's, 35.140'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3372.115'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.286'MB/sec write SST data time = 0.23's, 157.110'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 14403.973'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 69.945'MB/sec zip pipeline throughput = 81.016'MB/sec entries = 926095 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.93 avg-zval = 32.87 usrkeys = 926095 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.93 avg-zval = 32.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926095 bytes = 3357104 } seq expand size = 6482665 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5135 value = 3.5575 dict = 1.00 all = 3.6627 } Zip/UnZip{ index = 0.0951 value = 0.2811 dict = 1.00 all = 0.2730 } ---------------------------- total value len = 9.650101 GB avg = 0.164 KB (by entry num) total key len = 1.623171 GB avg = 0.028 KB total ukey len = 1.128128 GB avg = 0.016 KB total ukey num = 0.072062928 Billion total entry num = 0.058870657 Billion write speed all = 0.657415462 MB/sec (with seq num) write speed all = 0.629950513 MB/sec (without seq num) 2024/11/23-13:41:00.131447 139957173200448 [compaction_job.cc:1948] [default] [JOB 26959] Generated table #261557: 926095 keys, 36644538 bytes, temperature: kWarm 2024/11/23-13:41:00.131540 139957173200448 EVENT_LOG_v1 {"time_micros": 1732340460131491, "cf_name": "default", "job": 26959, "event": "table_file_creation", "file_number": 261557, "file_size": 36644538, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3485447251, "largest_seqno": 4000517932, "table_properties": {"data_size": 30439440, "index_size": 1761728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25930660, "raw_average_key_size": 28, "raw_value_size": 108287114, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926095, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340456, "oldest_key_time": 0, "file_creation_time": 1732340456, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261557, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:41:00.183903 139957173200448 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261558.sst) /dev/shm/ToplingTemp/Topling-62180-5IhjHX 2024/11/23-13:41:01.213744 139957173200448 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261558.sst): first pass time = 1.03's, 130.336'MB/sec 2024/11/23-13:41:02.773731 139957173200448 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261558.sst): old prealloc_size = 147639500, real_size = 32801744 2024/11/23-13:41:02.774276 139957173200448 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261558.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929854 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0308 GB avg-val = 116.34 avg-zval = 33.11 2024/11/23-13:41:02.775159 139957173200448 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261558.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:41:03.640266 139957173200448 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261558.sst): second pass time = 1.56's, 69.381'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.862'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3390.141'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.13's, 289.263'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33692.721'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.381'MB/sec zip pipeline throughput = 69.381'MB/sec entries = 929854 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.34 avg-zval = 33.11 usrkeys = 929854 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.34 avg-zval = 33.11 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929854 bytes = 3486960 } seq expand size = 6508978 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.4435 value = 3.5137 dict = 1.00 all = 3.6137 } Zip/UnZip{ index = 0.0958 value = 0.2846 dict = 1.00 all = 0.2767 } ---------------------------- total value len = 9.764792 GB avg = 0.163 KB (by entry num) total key len = 1.649207 GB avg = 0.028 KB total ukey len = 1.146725 GB avg = 0.016 KB total ukey num = 0.072992782 Billion total entry num = 0.059800511 Billion write speed all = 0.665483971 MB/sec (with seq num) write speed all = 0.637591008 MB/sec (without seq num) 2024/11/23-13:41:03.689798 139957173200448 [compaction_job.cc:1948] [default] [JOB 26959] Generated table #261558: 929854 keys, 37140974 bytes, temperature: kWarm 2024/11/23-13:41:03.689886 139957173200448 EVENT_LOG_v1 {"time_micros": 1732340463689839, "cf_name": "default", "job": 26959, "event": "table_file_creation", "file_number": 261558, "file_size": 37140974, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4000518605, "table_properties": {"data_size": 30788352, "index_size": 1780736, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26035912, "raw_average_key_size": 28, "raw_value_size": 108181956, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929854, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340460, "oldest_key_time": 0, "file_creation_time": 1732340460, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261558, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:41:03.740204 139957173200448 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261559.sst) /dev/shm/ToplingTemp/Topling-62180-xFDdbY 2024/11/23-13:41:04.986967 139957173200448 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261559.sst): first pass time = 1.25's, 107.658'MB/sec 2024/11/23-13:41:06.336542 139957173200448 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261559.sst): old prealloc_size = 147639500, real_size = 34628736 2024/11/23-13:41:06.337145 139957173200448 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261559.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929105 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0326 GB avg-val = 116.46 avg-zval = 35.12 2024/11/23-13:41:06.337964 139957173200448 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261559.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:41:07.288732 139957173200448 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261559.sst): second pass time = 1.35's, 80.227'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 40.801'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3052.271'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.282'MB/sec write SST data time = 0.13's, 305.256'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34810.518'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.227'MB/sec zip pipeline throughput = 80.227'MB/sec entries = 929105 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.46 avg-zval = 35.12 usrkeys = 929105 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.46 avg-zval = 35.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929105 bytes = 3368016 } seq expand size = 6503735 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0326 GB dict = 1.08 MB all = 0.0388 GB } UnZip/Zip{ index = 10.5125 value = 3.3162 dict = 1.00 all = 3.4548 } Zip/UnZip{ index = 0.0951 value = 0.3016 dict = 1.00 all = 0.2894 } ---------------------------- total value len = 9.879499 GB avg = 0.163 KB (by entry num) total key len = 1.675222 GB avg = 0.028 KB total ukey len = 1.165307 GB avg = 0.016 KB total ukey num = 0.073921887 Billion total entry num = 0.060729616 Billion write speed all = 0.673545350 MB/sec (with seq num) write speed all = 0.645225045 MB/sec (without seq num) 2024/11/23-13:41:07.337357 139957173200448 [compaction_job.cc:1948] [default] [JOB 26959] Generated table #261559: 929105 keys, 38849272 bytes, temperature: kWarm 2024/11/23-13:41:07.337417 139957173200448 EVENT_LOG_v1 {"time_micros": 1732340467337383, "cf_name": "default", "job": 26959, "event": "table_file_creation", "file_number": 261559, "file_size": 38849272, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4000518686, "table_properties": {"data_size": 32628656, "index_size": 1767616, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26014940, "raw_average_key_size": 28, "raw_value_size": 108202862, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929105, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340463, "oldest_key_time": 0, "file_creation_time": 1732340463, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261559, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:41:07.375236 139957173200448 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261560.sst) /dev/shm/ToplingTemp/Topling-62180-ADfkAY 2024/11/23-13:41:08.383049 139957173200448 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261560.sst): first pass time = 1.01's, 133.183'MB/sec 2024/11/23-13:41:09.670102 139957173200448 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261560.sst): old prealloc_size = 147639500, real_size = 31037168 2024/11/23-13:41:09.670663 139957173200448 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261560.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931405 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0290 GB avg-val = 116.10 avg-zval = 31.17 2024/11/23-13:41:09.672377 139957173200448 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261560.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:41:10.602802 139957173200448 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261560.sst): second pass time = 1.29's, 84.075'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 37.935'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3307.698'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.290'MB/sec write SST data time = 0.13's, 276.675'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34968.805'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.075'MB/sec zip pipeline throughput = 84.075'MB/sec entries = 931405 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.10 avg-zval = 31.17 usrkeys = 931405 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.10 avg-zval = 31.17 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931405 bytes = 3492784 } seq expand size = 6519835 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0290 GB dict = 1.08 MB all = 0.0354 GB } UnZip/Zip{ index = 10.5130 value = 3.7248 dict = 1.00 all = 3.7935 } Zip/UnZip{ index = 0.0951 value = 0.2685 dict = 1.00 all = 0.2636 } ---------------------------- total value len = 9.994157 GB avg = 0.162 KB (by entry num) total key len = 1.701301 GB avg = 0.028 KB total ukey len = 1.183935 GB avg = 0.016 KB total ukey num = 0.074853292 Billion total entry num = 0.061661021 Billion write speed all = 0.681617518 MB/sec (with seq num) write speed all = 0.652868421 MB/sec (without seq num) 2024/11/23-13:41:10.649721 139957173200448 [compaction_job.cc:1948] [default] [JOB 26959] Generated table #261560: 931405 keys, 35381296 bytes, temperature: kWarm 2024/11/23-13:41:10.649792 139957173200448 EVENT_LOG_v1 {"time_micros": 1732340470649755, "cf_name": "default", "job": 26959, "event": "table_file_creation", "file_number": 261560, "file_size": 35381296, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4000518465, "table_properties": {"data_size": 29032224, "index_size": 1771904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26079340, "raw_average_key_size": 28, "raw_value_size": 108138390, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931405, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340467, "oldest_key_time": 0, "file_creation_time": 1732340467, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261560, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:41:10.686292 139957173200448 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261561.sst) /dev/shm/ToplingTemp/Topling-62180-ptwoDW 2024/11/23-13:41:11.650406 139957173200448 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261561.sst): first pass time = 0.96's, 139.218'MB/sec 2024/11/23-13:41:12.947681 139957173200448 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261561.sst): old prealloc_size = 147639500, real_size = 31799088 2024/11/23-13:41:12.948281 139957173200448 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261561.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930680 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0298 GB avg-val = 116.21 avg-zval = 32.00 2024/11/23-13:41:12.950304 139957173200448 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261561.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:41:13.910553 139957173200448 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261561.sst): second pass time = 1.30's, 83.432'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 37.535'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3096.866'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.280'MB/sec write SST data time = 0.13's, 275.222'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33933.413'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.432'MB/sec zip pipeline throughput = 83.432'MB/sec entries = 930680 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.21 avg-zval = 32.00 usrkeys = 930680 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.21 avg-zval = 32.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930680 bytes = 3490064 } seq expand size = 6514760 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0298 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.4434 value = 3.6315 dict = 1.00 all = 3.7138 } Zip/UnZip{ index = 0.0958 value = 0.2754 dict = 1.00 all = 0.2693 } ---------------------------- total value len = 10.108830 GB avg = 0.162 KB (by entry num) total key len = 1.727360 GB avg = 0.028 KB total ukey len = 1.202549 GB avg = 0.016 KB total ukey num = 0.075783972 Billion total entry num = 0.062591701 Billion write speed all = 0.689686533 MB/sec (with seq num) write speed all = 0.660509137 MB/sec (without seq num) 2024/11/23-13:41:13.956194 139957173200448 [compaction_job.cc:1948] [default] [JOB 26959] Generated table #261561: 930680 keys, 36140624 bytes, temperature: kWarm 2024/11/23-13:41:13.956272 139957173200448 EVENT_LOG_v1 {"time_micros": 1732340473956227, "cf_name": "default", "job": 26959, "event": "table_file_creation", "file_number": 261561, "file_size": 36140624, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3419966228, "largest_seqno": 4000518503, "table_properties": {"data_size": 29783904, "index_size": 1782336, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26059040, "raw_average_key_size": 28, "raw_value_size": 108158816, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930680, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340470, "oldest_key_time": 0, "file_creation_time": 1732340470, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261561, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:41:14.001613 139957173200448 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261562.sst) /dev/shm/ToplingTemp/Topling-62180-WAYgOW 2024/11/23-13:41:14.455606 139957173200448 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261562.sst): first pass time = 0.45's, 230.516'MB/sec 2024/11/23-13:41:15.512087 139957173200448 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261562.sst): old prealloc_size = 147639500, real_size = 25169424 2024/11/23-13:41:15.512517 139957173200448 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261562.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 720540 prefix = 4 raw-key = 0.0115 GB zip-key = 0.0014 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0845 GB zip-val = 0.0236 GB avg-val = 117.23 avg-zval = 32.75 2024/11/23-13:41:15.513199 139957173200448 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261562.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0018 GB, waited 0.000 sec, Key+Value = 0.105 GB 2024/11/23-13:41:16.202999 139957173200448 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00/261562.sst): second pass time = 1.06's, 80.013'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.69's, 41.182'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3399.141'MB/sec (index lex order gen) rebuild zvType time = 0.58's, 0.311'MB/sec write SST data time = 0.11's, 257.266'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.847'MB, throughput = 25533.339'MB/sec zip my value time = 1.06's, unzip length = 0.084'GB zip my value throughput = 80.013'MB/sec zip pipeline throughput = 80.013'MB/sec entries = 720540 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.23 avg-zval = 32.75 usrkeys = 720540 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.23 avg-zval = 32.75 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 720540 bytes = 2611968 } seq expand size = 5043780 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0144 GB value = 0.0845 GB dict = 0.85 MB all = 0.1046 GB } __ZipSize{ index = 0.0014 GB value = 0.0236 GB dict = 0.85 MB all = 0.0285 GB } UnZip/Zip{ index = 10.3740 value = 3.5792 dict = 1.00 all = 3.6780 } Zip/UnZip{ index = 0.0964 value = 0.2794 dict = 1.00 all = 0.2719 } ---------------------------- total value len = 10.198343 GB avg = 0.161 KB (by entry num) total key len = 1.747535 GB avg = 0.028 KB total ukey len = 1.216960 GB avg = 0.016 KB total ukey num = 0.076504512 Billion total entry num = 0.063312241 Billion write speed all = 0.695984991 MB/sec (with seq num) write speed all = 0.666475654 MB/sec (without seq num) 2024/11/23-13:41:16.241355 139957173200448 [compaction_job.cc:1948] [default] [JOB 26959] Generated table #261562: 720540 keys, 28451348 bytes, temperature: kWarm 2024/11/23-13:41:16.241419 139957173200448 EVENT_LOG_v1 {"time_micros": 1732340476241385, "cf_name": "default", "job": 26959, "event": "table_file_creation", "file_number": 261562, "file_size": 28451348, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3642179108, "largest_seqno": 4000517274, "table_properties": {"data_size": 23600000, "index_size": 1389120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 20175120, "raw_average_key_size": 28, "raw_value_size": 84468816, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 720540, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340473, "oldest_key_time": 0, "file_creation_time": 1732340473, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261562, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:41:16.264786 139957173200448 [compaction_job.cc:703] job-26959: subcompact[0], size: 0.436125 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26959/att-00 [261551,261552,261553,261554,261555,261556,261557,261558,261559,261560,261561,261562] 2024/11/23-13:41:16.340649 139957173200448 (Original Log Time 2024/11/23-13:40:30.323022) EVENT_LOG_v1 {"time_micros": 1732340430322966, "job": 26959, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [261547], "files_L3": [261492, 261493, 261494, 261505, 261506, 261507, 261508, 261509, 261510, 261467, 261468, 261469, 261470], "score": 0.992957, "input_data_size": 455255514, "oldest_snapshot_seqno": 4000531485} 2024/11/23-13:41:16.340652 139957173200448 (Original Log Time 2024/11/23-13:41:16.339638) [compaction_job.cc:2022] [default] [JOB 26959] Compacted 1@2 + 13@3 files to L3 => 436125198 bytes 2024/11/23-13:41:16.340654 139957173200448 (Original Log Time 2024/11/23-13:41:16.340578) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 12 0 0 0] max score 0.02, MB/sec: 9.9 rd, 9.5 wr, level 3, files in(1, 13) out(12 +0 blob) MB in(25.2, 409.0 +0.0 blob) out(415.9 +0.0 blob), read-write-amplify(33.7) write-amplify(16.5) OK, records in: 10987352, records dropped: 33275 output_compression: Snappy 2024/11/23-13:41:16.340660 139957173200448 (Original Log Time 2024/11/23-13:41:16.340629) EVENT_LOG_v1 {"time_micros": 1732340476340608, "cf": "default", "job": 26959, "event": "compaction_finished", "compaction_time_micros": 45941772, "compaction_time_cpu_micros": 19947476, "output_level": 3, "num_output_files": 12, "total_output_size": 436125198, "num_input_records": 10987352, "num_output_records": 10954077, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 387214165, "file_range_sync_nanos": 0, "file_fsync_nanos": 9719, "file_prepare_write_nanos": 42566, "lsm_state": [0, 0, 0, 12, 0, 0, 0]} 2024/11/23-13:41:16.465317 139957173200448 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732340430,"executesMs":46157,"compactionJobId":26959,"attempt":0,"compactionInputRawBytes":1585764888,"compactionInputZipBytes":455255514,"compactionOutputRawBytes":0,"compactionOutputZipBytes":436125198,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-08a84b00d857dd168c976fae8a8c609b-554b4d30ce0738e2-00"}}, response = 2024/11/23-13:41:16.465394 139957173200448 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-26959/att-00: olev 3, work 46.157 s, result 7.692 ms, install 1.037 ms, input{raw 1.477 GiB zip 434.165 MiB}