2024/11/23-07:42:40.147216 140025196639808 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00: fork to child time = 0.009418 sec 2024/11/23-07:42:40.148127 140025196639808 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00 2024/11/23-07:42:40.155415 140025196639808 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00 2024/11/23-07:42:40.156527 140025196639808 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-07:42:40.156877 140025196639808 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-07:42:40.203853 140025196639808 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-22043/att-00/MANIFEST-206347 2024/11/23-07:42:40.232945 140025196639808 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-22043/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 250913, last_sequence is 3865805398, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 250765 2024/11/23-07:42:40.232954 140025196639808 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-07:42:40.233051 140025196639808 [version_set.cc:5721] Creating manifest 250913 2024/11/23-07:42:40.264906 140025196639808 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-07:42:40.290463 140025196639808 [compaction_job.cc:2437] [default] [JOB 22043] Compacting 1@2 + 14@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-07:42:40.290484 140025196639808 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 16539 Base level 2, inputs: [250903(27MB)], [250813(37MB) 250845(37MB) 250846(35MB) 250847(34MB) 250848(35MB) 250849(35MB) 250850(36MB) 250851(36MB) 250852(35MB) 250853(34MB) 250854(35MB) 250855(35MB) 250856(35MB) 250857(35MB)] 2024/11/23-07:42:40.296277 140025196639808 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-07:42:40.296309 140025196639808 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-07:42:40.296330 140025196639808 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-07:42:40.296335 140025196639808 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-07:42:40.296345 140025196639808 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-07:42:40.777605 140025196639808 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250914.sst) /dev/shm/ToplingTemp/Topling-450218-vxIOMB 2024/11/23-07:42:42.017569 140025196639808 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250914.sst): first pass time = 1.24's, 108.248'MB/sec 2024/11/23-07:42:45.828966 140025196639808 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250914.sst): old prealloc_size = 147639500, real_size = 34764672 2024/11/23-07:42:45.829553 140025196639808 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250914.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927472 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0328 GB avg-val = 116.71 avg-zval = 35.32 2024/11/23-07:42:45.830371 140025196639808 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250914.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:42:47.001757 140025196639808 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250914.sst): second pass time = 3.81's, 28.406'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.17's, 33.140'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3165.957'MB/sec (index lex order gen) rebuild zvType time = 0.93's, 0.248'MB/sec write SST data time = 0.24's, 163.525'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 26899.057'MB/sec zip my value time = 3.81's, unzip length = 0.108'GB zip my value throughput = 28.406'MB/sec zip pipeline throughput = 124.934'MB/sec entries = 927472 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.71 avg-zval = 35.32 usrkeys = 927472 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.71 avg-zval = 35.32 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927472 bytes = 3246160 } seq expand size = 6492304 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0328 GB dict = 1.08 MB all = 0.0389 GB } UnZip/Zip{ index = 10.4441 value = 3.3046 dict = 1.00 all = 3.4534 } Zip/UnZip{ index = 0.0957 value = 0.3026 dict = 1.00 all = 0.2896 } ---------------------------- total value len = 0.114741 GB avg = 0.124 KB (by entry num) total key len = 0.025969 GB avg = 0.028 KB total ukey len = 0.018549 GB avg = 0.020 KB total ukey num = 0.000927472 Billion total entry num = 0.000927472 Billion write speed all = 0.001711303 MB/sec (with seq num) write speed all = 0.001621065 MB/sec (without seq num) 2024/11/23-07:42:47.042480 140025196639808 [compaction_job.cc:1948] [default] [JOB 22043] Generated table #250914: 927472 keys, 38865410 bytes, temperature: kWarm 2024/11/23-07:42:47.042589 140025196639808 EVENT_LOG_v1 {"time_micros": 1732318967042519, "cf_name": "default", "job": 22043, "event": "table_file_creation", "file_number": 250914, "file_size": 38865410, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3661838825, "largest_seqno": 3865786223, "table_properties": {"data_size": 32756528, "index_size": 1776064, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25969216, "raw_average_key_size": 28, "raw_value_size": 108248532, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927472, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732318960, "oldest_key_time": 0, "file_creation_time": 1732318960, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250914, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:42:47.137617 140025196639808 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250915.sst) /dev/shm/ToplingTemp/Topling-450218-Q9YBBz 2024/11/23-07:42:48.670049 140025196639808 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250915.sst): first pass time = 1.53's, 87.588'MB/sec 2024/11/23-07:42:51.234635 140025196639808 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250915.sst): old prealloc_size = 147639500, real_size = 35537280 2024/11/23-07:42:51.235199 140025196639808 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250915.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924912 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0335 GB avg-val = 117.11 avg-zval = 36.26 2024/11/23-07:42:51.235960 140025196639808 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250915.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:42:52.722798 140025196639808 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250915.sst): second pass time = 2.56's, 42.250'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.49's, 26.553'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3250.812'MB/sec (index lex order gen) rebuild zvType time = 1.24's, 0.187'MB/sec write SST data time = 0.25's, 158.563'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33501.782'MB/sec zip my value time = 2.56's, unzip length = 0.108'GB zip my value throughput = 42.250'MB/sec zip pipeline throughput = 129.888'MB/sec entries = 924912 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.11 avg-zval = 36.26 usrkeys = 924912 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.11 avg-zval = 36.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924912 bytes = 3121600 } seq expand size = 6474384 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0335 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.4458 value = 3.2301 dict = 1.00 all = 3.3967 } Zip/UnZip{ index = 0.0957 value = 0.3096 dict = 1.00 all = 0.2944 } ---------------------------- total value len = 0.229535 GB avg = 0.124 KB (by entry num) total key len = 0.051867 GB avg = 0.028 KB total ukey len = 0.037048 GB avg = 0.020 KB total ukey num = 0.001852384 Billion total entry num = 0.001852384 Billion write speed all = 0.003422151 MB/sec (with seq num) write speed all = 0.003241935 MB/sec (without seq num) 2024/11/23-07:42:52.765403 140025196639808 [compaction_job.cc:1948] [default] [JOB 22043] Generated table #250915: 924912 keys, 39514222 bytes, temperature: kWarm 2024/11/23-07:42:52.765483 140025196639808 EVENT_LOG_v1 {"time_micros": 1732318972765440, "cf_name": "default", "job": 22043, "event": "table_file_creation", "file_number": 250915, "file_size": 39514222, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3759264366, "largest_seqno": 3865786633, "table_properties": {"data_size": 33534960, "index_size": 1770880, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25897536, "raw_average_key_size": 28, "raw_value_size": 108320236, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924912, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732318967, "oldest_key_time": 0, "file_creation_time": 1732318967, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250915, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:42:52.815044 140025196639808 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250916.sst) /dev/shm/ToplingTemp/Topling-450218-7Af1oy 2024/11/23-07:42:53.802455 140025196639808 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250916.sst): first pass time = 0.99's, 135.935'MB/sec 2024/11/23-07:42:55.704950 140025196639808 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250916.sst): old prealloc_size = 147639500, real_size = 33259664 2024/11/23-07:42:55.705621 140025196639808 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250916.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928389 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.57 avg-zval = 33.65 2024/11/23-07:42:55.706504 140025196639808 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250916.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:42:56.873506 140025196639808 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250916.sst): second pass time = 1.90's, 56.909'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.17's, 32.072'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2752.931'MB/sec (index lex order gen) rebuild zvType time = 1.02's, 0.228'MB/sec write SST data time = 0.15's, 252.416'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30950.129'MB/sec zip my value time = 1.90's, unzip length = 0.108'GB zip my value throughput = 56.909'MB/sec zip pipeline throughput = 72.932'MB/sec entries = 928389 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.57 avg-zval = 33.65 usrkeys = 928389 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.57 avg-zval = 33.65 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928389 bytes = 3365424 } seq expand size = 6498723 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.3789 value = 3.4644 dict = 1.00 all = 3.5812 } Zip/UnZip{ index = 0.0963 value = 0.2886 dict = 1.00 all = 0.2792 } ---------------------------- total value len = 0.344257 GB avg = 0.124 KB (by entry num) total key len = 0.077862 GB avg = 0.028 KB total ukey len = 0.055615 GB avg = 0.020 KB total ukey num = 0.002780773 Billion total entry num = 0.002780773 Billion write speed all = 0.005133155 MB/sec (with seq num) write speed all = 0.004862632 MB/sec (without seq num) 2024/11/23-07:42:56.934284 140025196639808 [compaction_job.cc:1948] [default] [JOB 22043] Generated table #250916: 928389 keys, 37478586 bytes, temperature: kWarm 2024/11/23-07:42:56.934347 140025196639808 EVENT_LOG_v1 {"time_micros": 1732318976934316, "cf_name": "default", "job": 22043, "event": "table_file_creation", "file_number": 250916, "file_size": 37478586, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038106, "largest_seqno": 3865785783, "table_properties": {"data_size": 31238400, "index_size": 1788992, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25994892, "raw_average_key_size": 28, "raw_value_size": 108222930, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928389, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732318972, "oldest_key_time": 0, "file_creation_time": 1732318972, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250916, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:42:56.973718 140025196639808 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250917.sst) /dev/shm/ToplingTemp/Topling-450218-tI6FFB 2024/11/23-07:42:57.824207 140025196639808 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250917.sst): first pass time = 0.85's, 157.820'MB/sec 2024/11/23-07:42:59.730979 140025196639808 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250917.sst): old prealloc_size = 147639500, real_size = 31415200 2024/11/23-07:42:59.731639 140025196639808 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250917.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926826 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0294 GB avg-val = 116.81 avg-zval = 31.73 2024/11/23-07:42:59.732483 140025196639808 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250917.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:43:00.751273 140025196639808 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250917.sst): second pass time = 1.90's, 56.933'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.02's, 34.807'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2791.008'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.287'MB/sec write SST data time = 0.21's, 166.881'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35368.652'MB/sec zip my value time = 1.90's, unzip length = 0.108'GB zip my value throughput = 56.933'MB/sec zip pipeline throughput = 93.712'MB/sec entries = 926826 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.81 avg-zval = 31.73 usrkeys = 926826 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.81 avg-zval = 31.73 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926826 bytes = 3243904 } seq expand size = 6487782 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4417 value = 3.6815 dict = 1.00 all = 3.7793 } Zip/UnZip{ index = 0.0958 value = 0.2716 dict = 1.00 all = 0.2646 } ---------------------------- total value len = 0.459011 GB avg = 0.124 KB (by entry num) total key len = 0.103813 GB avg = 0.028 KB total ukey len = 0.074152 GB avg = 0.020 KB total ukey num = 0.003707599 Billion total entry num = 0.003707599 Billion write speed all = 0.006843875 MB/sec (with seq num) write speed all = 0.006483204 MB/sec (without seq num) 2024/11/23-07:43:00.787095 140025196639808 [compaction_job.cc:1948] [default] [JOB 22043] Generated table #250917: 926826 keys, 35513490 bytes, temperature: kWarm 2024/11/23-07:43:00.787155 140025196639808 EVENT_LOG_v1 {"time_micros": 1732318980787125, "cf_name": "default", "job": 22043, "event": "table_file_creation", "file_number": 250917, "file_size": 35513490, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3724004287, "largest_seqno": 3865786396, "table_properties": {"data_size": 29408080, "index_size": 1775232, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25951128, "raw_average_key_size": 28, "raw_value_size": 108266608, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926826, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732318976, "oldest_key_time": 0, "file_creation_time": 1732318976, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250917, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:43:00.825990 140025196639808 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250918.sst) /dev/shm/ToplingTemp/Topling-450218-o17qVy 2024/11/23-07:43:01.643907 140025196639808 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250918.sst): first pass time = 0.82's, 164.105'MB/sec 2024/11/23-07:43:03.447575 140025196639808 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250918.sst): old prealloc_size = 147639500, real_size = 32511072 2024/11/23-07:43:03.448134 140025196639808 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250918.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928830 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0305 GB avg-val = 116.50 avg-zval = 32.85 2024/11/23-07:43:03.449021 140025196639808 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250918.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:43:04.358309 140025196639808 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250918.sst): second pass time = 1.80's, 60.019'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 40.332'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3285.567'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.298'MB/sec write SST data time = 0.13's, 282.882'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35512.600'MB/sec zip my value time = 1.80's, unzip length = 0.108'GB zip my value throughput = 60.019'MB/sec zip pipeline throughput = 98.931'MB/sec entries = 928830 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.50 avg-zval = 32.85 usrkeys = 928830 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.50 avg-zval = 32.85 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928830 bytes = 3367024 } seq expand size = 6501810 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5144 value = 3.5465 dict = 1.00 all = 3.6541 } Zip/UnZip{ index = 0.0951 value = 0.2820 dict = 1.00 all = 0.2737 } ---------------------------- total value len = 0.573724 GB avg = 0.124 KB (by entry num) total key len = 0.129820 GB avg = 0.028 KB total ukey len = 0.092729 GB avg = 0.020 KB total ukey num = 0.004636429 Billion total entry num = 0.004636429 Billion write speed all = 0.008554634 MB/sec (with seq num) write speed all = 0.008103627 MB/sec (without seq num) 2024/11/23-07:43:04.403250 140025196639808 [compaction_job.cc:1948] [default] [JOB 22043] Generated table #250918: 928830 keys, 36731176 bytes, temperature: kWarm 2024/11/23-07:43:04.403316 140025196639808 EVENT_LOG_v1 {"time_micros": 1732318984403281, "cf_name": "default", "job": 22043, "event": "table_file_creation", "file_number": 250918, "file_size": 36731176, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038136, "largest_seqno": 3865785844, "table_properties": {"data_size": 30511888, "index_size": 1766784, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26007240, "raw_average_key_size": 28, "raw_value_size": 108210584, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928830, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732318980, "oldest_key_time": 0, "file_creation_time": 1732318980, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250918, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:43:04.440611 140025196639808 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250919.sst) /dev/shm/ToplingTemp/Topling-450218-zoGKqx 2024/11/23-07:43:05.463011 140025196639808 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250919.sst): first pass time = 1.02's, 131.282'MB/sec 2024/11/23-07:43:06.996684 140025196639808 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250919.sst): old prealloc_size = 147639500, real_size = 33631536 2024/11/23-07:43:06.997232 140025196639808 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250919.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924228 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 117.22 avg-zval = 34.24 2024/11/23-07:43:06.998017 140025196639808 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250919.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:43:07.937605 140025196639808 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250919.sst): second pass time = 1.53's, 70.667'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 40.092'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3316.805'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.284'MB/sec write SST data time = 0.13's, 296.026'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 37288.915'MB/sec zip my value time = 1.53's, unzip length = 0.108'GB zip my value throughput = 70.667'MB/sec zip pipeline throughput = 118.363'MB/sec entries = 924228 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.22 avg-zval = 34.24 usrkeys = 924228 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.22 avg-zval = 34.24 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924228 bytes = 3234816 } seq expand size = 6469596 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5106 value = 3.4240 dict = 1.00 all = 3.5580 } Zip/UnZip{ index = 0.0951 value = 0.2921 dict = 1.00 all = 0.2811 } ---------------------------- total value len = 0.688533 GB avg = 0.124 KB (by entry num) total key len = 0.155698 GB avg = 0.028 KB total ukey len = 0.111213 GB avg = 0.020 KB total ukey num = 0.005560657 Billion total entry num = 0.005560657 Billion write speed all = 0.010264854 MB/sec (with seq num) write speed all = 0.009723966 MB/sec (without seq num) 2024/11/23-07:43:07.975116 140025196639808 [compaction_job.cc:1948] [default] [JOB 22043] Generated table #250919: 924228 keys, 37722994 bytes, temperature: kWarm 2024/11/23-07:43:07.975180 140025196639808 EVENT_LOG_v1 {"time_micros": 1732318987975149, "cf_name": "default", "job": 22043, "event": "table_file_creation", "file_number": 250919, "file_size": 37722994, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3707111735, "largest_seqno": 3865785502, "table_properties": {"data_size": 31641632, "index_size": 1758656, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25878384, "raw_average_key_size": 28, "raw_value_size": 108339436, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924228, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732318984, "oldest_key_time": 0, "file_creation_time": 1732318984, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250919, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:43:08.015089 140025196639808 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250920.sst) /dev/shm/ToplingTemp/Topling-450218-HxXm1x 2024/11/23-07:43:08.782593 140025196639808 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250920.sst): first pass time = 0.77's, 174.885'MB/sec 2024/11/23-07:43:10.473509 140025196639808 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250920.sst): old prealloc_size = 147639500, real_size = 36042592 2024/11/23-07:43:10.474072 140025196639808 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250920.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931787 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0340 GB avg-val = 116.04 avg-zval = 36.50 2024/11/23-07:43:10.474435 140025196639808 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250920.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:43:11.194519 140025196639808 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250920.sst): second pass time = 1.69's, 63.974'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.72's, 52.932'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3305.789'MB/sec (index lex order gen) rebuild zvType time = 0.59's, 0.393'MB/sec write SST data time = 0.13's, 297.828'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 28408.487'MB/sec zip my value time = 1.69's, unzip length = 0.108'GB zip my value throughput = 63.974'MB/sec zip pipeline throughput = 123.803'MB/sec entries = 931787 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.04 avg-zval = 36.50 usrkeys = 931787 avg-key = 20.00 avg-zkey = 1.93 avg-val = 118.29 avg-zval = 36.50 TagRS{ kinds = RS_Key0_TagN bytes = 145680 } TagArray{ size = 298977 bytes = 1121184 } seq expand size = 2092839 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0340 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.3761 value = 3.1790 dict = 1.00 all = 3.5169 } Zip/UnZip{ index = 0.0964 value = 0.3146 dict = 1.00 all = 0.2843 } ---------------------------- total value len = 0.798753 GB avg = 0.123 KB (by entry num) total key len = 0.181788 GB avg = 0.028 KB total ukey len = 0.129849 GB avg = 0.020 KB total ukey num = 0.006492444 Billion total entry num = 0.006492444 Billion write speed all = 0.011921758 MB/sec (with seq num) write speed all = 0.011290259 MB/sec (without seq num) 2024/11/23-07:43:11.230763 140025196639808 [compaction_job.cc:1948] [default] [JOB 22043] Generated table #250920: 931787 keys, 38163913 bytes, temperature: kWarm 2024/11/23-07:43:11.230832 140025196639808 EVENT_LOG_v1 {"time_micros": 1732318991230799, "cf_name": "default", "job": 22043, "event": "table_file_creation", "file_number": 250920, "file_size": 38163913, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3865783642, "table_properties": {"data_size": 34013456, "index_size": 1796032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26090036, "raw_average_key_size": 28, "raw_value_size": 108127702, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931787, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732318988, "oldest_key_time": 0, "file_creation_time": 1732318988, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250920, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:43:11.266760 140025196639808 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250921.sst) /dev/shm/ToplingTemp/Topling-450218-n3Ju6A 2024/11/23-07:43:12.036143 140025196639808 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250921.sst): first pass time = 0.77's, 174.466'MB/sec 2024/11/23-07:43:13.754476 140025196639808 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250921.sst): old prealloc_size = 147639500, real_size = 36371936 2024/11/23-07:43:13.755012 140025196639808 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250921.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926831 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0344 GB avg-val = 116.81 avg-zval = 37.10 2024/11/23-07:43:13.755057 140025196639808 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250921.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:43:14.398533 140025196639808 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250921.sst): second pass time = 1.72's, 63.034'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 57.807'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3410.656'MB/sec (index lex order gen) rebuild zvType time = 0.52's, 0.446'MB/sec write SST data time = 0.12's, 300.092'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 29773.313'MB/sec zip my value time = 1.72's, unzip length = 0.108'GB zip my value throughput = 63.034'MB/sec zip pipeline throughput = 80.800'MB/sec entries = 926831 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.81 avg-zval = 37.10 usrkeys = 926831 avg-key = 20.00 avg-zkey = 1.89 avg-val = 116.81 avg-zval = 37.10 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5834 value = 3.1483 dict = 1.00 all = 3.6051 } Zip/UnZip{ index = 0.0945 value = 0.3176 dict = 1.00 all = 0.2774 } ---------------------------- total value len = 0.907020 GB avg = 0.122 KB (by entry num) total key len = 0.207740 GB avg = 0.028 KB total ukey len = 0.148386 GB avg = 0.020 KB total ukey num = 0.007419275 Billion total entry num = 0.007419275 Billion write speed all = 0.013553094 MB/sec (with seq num) write speed all = 0.012831474 MB/sec (without seq num) 2024/11/23-07:43:14.435651 140025196639808 [compaction_job.cc:1948] [default] [JOB 22043] Generated table #250921: 926831 keys, 37230112 bytes, temperature: kWarm 2024/11/23-07:43:14.435722 140025196639808 EVENT_LOG_v1 {"time_micros": 1732318994435683, "cf_name": "default", "job": 22043, "event": "table_file_creation", "file_number": 250921, "file_size": 37230112, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34388560, "index_size": 1751488, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25951268, "raw_average_key_size": 28, "raw_value_size": 108266502, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926831, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732318991, "oldest_key_time": 0, "file_creation_time": 1732318991, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250921, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:43:14.476943 140025196639808 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250922.sst) /dev/shm/ToplingTemp/Topling-450218-nH6jXx 2024/11/23-07:43:15.296597 140025196639808 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250922.sst): first pass time = 0.82's, 163.757'MB/sec 2024/11/23-07:43:16.777153 140025196639808 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250922.sst): old prealloc_size = 147639500, real_size = 32303920 2024/11/23-07:43:16.777748 140025196639808 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250922.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929523 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1082 GB zip-val = 0.0303 GB avg-val = 116.39 avg-zval = 32.61 2024/11/23-07:43:16.778607 140025196639808 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250922.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:43:17.679933 140025196639808 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250922.sst): second pass time = 1.48's, 73.112'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 40.464'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3062.597'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.304'MB/sec write SST data time = 0.14's, 263.825'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33198.290'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.112'MB/sec zip pipeline throughput = 73.112'MB/sec entries = 929523 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.39 avg-zval = 32.61 usrkeys = 929523 avg-key = 20.00 avg-zkey = 1.89 avg-val = 122.87 avg-zval = 32.61 TagRS{ kinds = RS_Key0_TagN bytes = 145320 } TagArray{ size = 859673 bytes = 3223792 } seq expand size = 6017711 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5789 value = 3.5690 dict = 1.00 all = 3.6742 } Zip/UnZip{ index = 0.0945 value = 0.2802 dict = 1.00 all = 0.2722 } ---------------------------- total value len = 1.021229 GB avg = 0.122 KB (by entry num) total key len = 0.233766 GB avg = 0.028 KB total ukey len = 0.166976 GB avg = 0.020 KB total ukey num = 0.008348798 Billion total entry num = 0.008348798 Billion write speed all = 0.015257450 MB/sec (with seq num) write speed all = 0.014445454 MB/sec (without seq num) 2024/11/23-07:43:17.719675 140025196639808 [compaction_job.cc:1948] [default] [JOB 22043] Generated table #250922: 929523 keys, 36529568 bytes, temperature: kWarm 2024/11/23-07:43:17.719736 140025196639808 EVENT_LOG_v1 {"time_micros": 1732318997719704, "cf_name": "default", "job": 22043, "event": "table_file_creation", "file_number": 250922, "file_size": 36529568, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3865786566, "table_properties": {"data_size": 30314016, "index_size": 1757312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26026644, "raw_average_key_size": 28, "raw_value_size": 108191178, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929523, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732318994, "oldest_key_time": 0, "file_creation_time": 1732318994, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250922, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:43:17.758840 140025196639808 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250923.sst) /dev/shm/ToplingTemp/Topling-450218-iGdxiz 2024/11/23-07:43:18.593158 140025196639808 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250923.sst): first pass time = 0.83's, 160.879'MB/sec 2024/11/23-07:43:19.894600 140025196639808 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250923.sst): old prealloc_size = 147639500, real_size = 30713216 2024/11/23-07:43:19.895161 140025196639808 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250923.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927588 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0287 GB avg-val = 116.70 avg-zval = 30.93 2024/11/23-07:43:19.895950 140025196639808 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250923.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:43:20.853863 140025196639808 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250923.sst): second pass time = 1.30's, 83.216'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 36.412'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3372.132'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.285'MB/sec write SST data time = 0.15's, 239.046'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 27370.525'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.216'MB/sec zip pipeline throughput = 83.216'MB/sec entries = 927588 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.70 avg-zval = 30.93 usrkeys = 927588 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.70 avg-zval = 30.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927588 bytes = 3362528 } seq expand size = 6493116 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0287 GB dict = 1.08 MB all = 0.0349 GB } UnZip/Zip{ index = 10.3770 value = 3.7725 dict = 1.00 all = 3.8427 } Zip/UnZip{ index = 0.0964 value = 0.2651 dict = 1.00 all = 0.2602 } ---------------------------- total value len = 1.135967 GB avg = 0.122 KB (by entry num) total key len = 0.259739 GB avg = 0.028 KB total ukey len = 0.185528 GB avg = 0.020 KB total ukey num = 0.009276386 Billion total entry num = 0.009276386 Billion write speed all = 0.016967470 MB/sec (with seq num) write speed all = 0.016065293 MB/sec (without seq num) 2024/11/23-07:43:20.888599 140025196639808 [compaction_job.cc:1948] [default] [JOB 22043] Generated table #250923: 927588 keys, 34928038 bytes, temperature: kWarm 2024/11/23-07:43:20.888668 140025196639808 EVENT_LOG_v1 {"time_micros": 1732319000888637, "cf_name": "default", "job": 22043, "event": "table_file_creation", "file_number": 250923, "file_size": 34928038, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3523475373, "largest_seqno": 3865785566, "table_properties": {"data_size": 28693360, "index_size": 1787776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25972464, "raw_average_key_size": 28, "raw_value_size": 108245268, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927588, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732318997, "oldest_key_time": 0, "file_creation_time": 1732318997, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250923, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:43:20.922909 140025196639808 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250924.sst) /dev/shm/ToplingTemp/Topling-450218-RhYJgA 2024/11/23-07:43:21.771957 140025196639808 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250924.sst): first pass time = 0.85's, 158.087'MB/sec 2024/11/23-07:43:23.124454 140025196639808 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250924.sst): old prealloc_size = 147639500, real_size = 33249696 2024/11/23-07:43:23.125014 140025196639808 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250924.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930116 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.30 avg-zval = 33.60 2024/11/23-07:43:23.125770 140025196639808 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250924.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:43:24.136360 140025196639808 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250924.sst): second pass time = 1.35's, 80.036'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.01's, 36.918'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3279.405'MB/sec (index lex order gen) rebuild zvType time = 0.88's, 0.263'MB/sec write SST data time = 0.13's, 290.762'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 36247.438'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.036'MB/sec zip pipeline throughput = 80.036'MB/sec entries = 930116 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.30 avg-zval = 33.60 usrkeys = 930116 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.30 avg-zval = 33.60 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930116 bytes = 3255424 } seq expand size = 6510812 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5122 value = 3.4619 dict = 1.00 all = 3.5928 } Zip/UnZip{ index = 0.0951 value = 0.2889 dict = 1.00 all = 0.2783 } ---------------------------- total value len = 1.250653 GB avg = 0.123 KB (by entry num) total key len = 0.285782 GB avg = 0.028 KB total ukey len = 0.204130 GB avg = 0.020 KB total ukey num = 0.010206502 Billion total entry num = 0.010206502 Billion write speed all = 0.018677549 MB/sec (with seq num) write speed all = 0.017684953 MB/sec (without seq num) 2024/11/23-07:43:24.174161 140025196639808 [compaction_job.cc:1948] [default] [JOB 22043] Generated table #250924: 930116 keys, 37357176 bytes, temperature: kWarm 2024/11/23-07:43:24.174224 140025196639808 EVENT_LOG_v1 {"time_micros": 1732319004174192, "cf_name": "default", "job": 22043, "event": "table_file_creation", "file_number": 250924, "file_size": 37357176, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3631595348, "largest_seqno": 3865786472, "table_properties": {"data_size": 31247376, "index_size": 1769600, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26043248, "raw_average_key_size": 28, "raw_value_size": 108174524, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930116, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732319000, "oldest_key_time": 0, "file_creation_time": 1732319000, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250924, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:43:24.209935 140025196639808 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250925.sst) /dev/shm/ToplingTemp/Topling-450218-qvO8kz 2024/11/23-07:43:25.045790 140025196639808 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250925.sst): first pass time = 0.84's, 160.583'MB/sec 2024/11/23-07:43:26.427750 140025196639808 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250925.sst): old prealloc_size = 147639500, real_size = 32469168 2024/11/23-07:43:26.428293 140025196639808 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250925.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929155 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0305 GB avg-val = 116.45 avg-zval = 32.78 2024/11/23-07:43:26.429083 140025196639808 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250925.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:43:27.440675 140025196639808 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250925.sst): second pass time = 1.38's, 78.362'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.01's, 36.223'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3458.152'MB/sec (index lex order gen) rebuild zvType time = 0.88's, 0.263'MB/sec write SST data time = 0.13's, 284.687'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 31950.386'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.362'MB/sec zip pipeline throughput = 78.362'MB/sec entries = 929155 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.45 avg-zval = 32.78 usrkeys = 929155 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.45 avg-zval = 32.78 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929155 bytes = 3368208 } seq expand size = 6504085 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.4450 value = 3.5525 dict = 1.00 all = 3.6581 } Zip/UnZip{ index = 0.0957 value = 0.2815 dict = 1.00 all = 0.2734 } ---------------------------- total value len = 1.365358 GB avg = 0.123 KB (by entry num) total key len = 0.311798 GB avg = 0.028 KB total ukey len = 0.222713 GB avg = 0.020 KB total ukey num = 0.011135657 Billion total entry num = 0.011135657 Billion write speed all = 0.020387405 MB/sec (with seq num) write speed all = 0.019304490 MB/sec (without seq num) 2024/11/23-07:43:27.476939 140025196639808 [compaction_job.cc:1948] [default] [JOB 22043] Generated table #250925: 929155 keys, 36690404 bytes, temperature: kWarm 2024/11/23-07:43:27.477003 140025196639808 EVENT_LOG_v1 {"time_micros": 1732319007476971, "cf_name": "default", "job": 22043, "event": "table_file_creation", "file_number": 250925, "file_size": 36690404, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 3865785594, "table_properties": {"data_size": 30457568, "index_size": 1779136, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26016340, "raw_average_key_size": 28, "raw_value_size": 108201418, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929155, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732319004, "oldest_key_time": 0, "file_creation_time": 1732319004, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250925, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:43:27.511935 140025196639808 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250926.sst) /dev/shm/ToplingTemp/Topling-450218-sEHEXz 2024/11/23-07:43:28.325361 140025196639808 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250926.sst): first pass time = 0.81's, 165.011'MB/sec 2024/11/23-07:43:29.675642 140025196639808 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250926.sst): old prealloc_size = 147639500, real_size = 31942416 2024/11/23-07:43:29.676189 140025196639808 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250926.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929154 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0299 GB avg-val = 116.45 avg-zval = 32.20 2024/11/23-07:43:29.676954 140025196639808 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250926.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:43:30.667365 140025196639808 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250926.sst): second pass time = 1.35's, 80.189'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.99's, 36.349'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3395.864'MB/sec (index lex order gen) rebuild zvType time = 0.86's, 0.270'MB/sec write SST data time = 0.13's, 278.210'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34684.789'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.189'MB/sec zip pipeline throughput = 80.189'MB/sec entries = 929154 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.45 avg-zval = 32.20 usrkeys = 929154 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.45 avg-zval = 32.20 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929154 bytes = 3252048 } seq expand size = 6504078 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.3756 value = 3.6165 dict = 1.00 all = 3.7234 } Zip/UnZip{ index = 0.0964 value = 0.2765 dict = 1.00 all = 0.2686 } ---------------------------- total value len = 1.480064 GB avg = 0.123 KB (by entry num) total key len = 0.337815 GB avg = 0.028 KB total ukey len = 0.241296 GB avg = 0.020 KB total ukey num = 0.012064811 Billion total entry num = 0.012064811 Billion write speed all = 0.022097143 MB/sec (with seq num) write speed all = 0.020923917 MB/sec (without seq num) 2024/11/23-07:43:30.699301 140025196639808 [compaction_job.cc:1948] [default] [JOB 22043] Generated table #250926: 929154 keys, 36047540 bytes, temperature: kWarm 2024/11/23-07:43:30.699362 140025196639808 EVENT_LOG_v1 {"time_micros": 1732319010699331, "cf_name": "default", "job": 22043, "event": "table_file_creation", "file_number": 250926, "file_size": 36047540, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3621075956, "largest_seqno": 3865786135, "table_properties": {"data_size": 29918912, "index_size": 1791040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26016312, "raw_average_key_size": 28, "raw_value_size": 108201452, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929154, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732319007, "oldest_key_time": 0, "file_creation_time": 1732319007, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250926, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:43:30.734095 140025196639808 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250927.sst) /dev/shm/ToplingTemp/Topling-450218-mDnTyz 2024/11/23-07:43:31.654737 140025196639808 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250927.sst): first pass time = 0.92's, 145.793'MB/sec 2024/11/23-07:43:32.966687 140025196639808 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250927.sst): old prealloc_size = 147639500, real_size = 32495024 2024/11/23-07:43:32.967270 140025196639808 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250927.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928846 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0305 GB avg-val = 116.50 avg-zval = 32.82 2024/11/23-07:43:32.968065 140025196639808 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250927.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:43:33.969179 140025196639808 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250927.sst): second pass time = 1.31's, 82.519'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.00's, 36.625'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3175.767'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.266'MB/sec write SST data time = 0.13's, 281.794'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35875.316'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.519'MB/sec zip pipeline throughput = 82.519'MB/sec entries = 928846 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.50 avg-zval = 32.82 usrkeys = 928846 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.50 avg-zval = 32.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928846 bytes = 3367088 } seq expand size = 6501922 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.4449 value = 3.5497 dict = 1.00 all = 3.6556 } Zip/UnZip{ index = 0.0957 value = 0.2817 dict = 1.00 all = 0.2736 } ---------------------------- total value len = 1.594776 GB avg = 0.123 KB (by entry num) total key len = 0.363822 GB avg = 0.028 KB total ukey len = 0.259873 GB avg = 0.020 KB total ukey num = 0.012993657 Billion total entry num = 0.012993657 Billion write speed all = 0.023806701 MB/sec (with seq num) write speed all = 0.022543201 MB/sec (without seq num) 2024/11/23-07:43:34.001115 140025196639808 [compaction_job.cc:1948] [default] [JOB 22043] Generated table #250927: 928846 keys, 36715288 bytes, temperature: kWarm 2024/11/23-07:43:34.001181 140025196639808 EVENT_LOG_v1 {"time_micros": 1732319014001146, "cf_name": "default", "job": 22043, "event": "table_file_creation", "file_number": 250927, "file_size": 36715288, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3864968253, "table_properties": {"data_size": 30484064, "index_size": 1778560, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26007688, "raw_average_key_size": 28, "raw_value_size": 108210128, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928846, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732319010, "oldest_key_time": 0, "file_creation_time": 1732319010, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250927, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:43:34.034561 140025196639808 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250928.sst) /dev/shm/ToplingTemp/Topling-450218-bhXr7A 2024/11/23-07:43:34.139896 140025196639808 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250928.sst): first pass time = 0.11's, 237.567'MB/sec 2024/11/23-07:43:34.381244 140025196639808 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250928.sst): old prealloc_size = 147639500, real_size = 7206048 2024/11/23-07:43:34.381394 140025196639808 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250928.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 171335 prefix = 4 raw-key = 0.0027 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0202 GB zip-val = 0.0068 GB avg-val = 118.00 avg-zval = 39.87 2024/11/23-07:43:34.381574 140025196639808 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250928.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0004 GB, waited 0.000 sec, Key+Value = 0.025 GB 2024/11/23-07:43:34.547057 140025196639808 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00/250928.sst): second pass time = 0.24's, 83.975'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 47.855'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2608.103'MB/sec (index lex order gen) rebuild zvType time = 0.14's, 0.303'MB/sec write SST data time = 0.02's, 326.264'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.213'MB, throughput = 7294.295'MB/sec zip my value time = 0.24's, unzip length = 0.020'GB zip my value throughput = 83.975'MB/sec zip pipeline throughput = 83.975'MB/sec entries = 171335 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 39.87 usrkeys = 171335 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 39.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 171335 bytes = 556848 } seq expand size = 1199345 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0034 GB value = 0.0202 GB dict = 0.21 MB all = 0.0250 GB } __ZipSize{ index = 0.0003 GB value = 0.0068 GB dict = 0.21 MB all = 0.0079 GB } UnZip/Zip{ index = 10.3383 value = 2.9594 dict = 1.00 all = 3.1528 } Zip/UnZip{ index = 0.0967 value = 0.3379 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 1.616193 GB avg = 0.123 KB (by entry num) total key len = 0.368620 GB avg = 0.028 KB total ukey len = 0.263300 GB avg = 0.020 KB total ukey num = 0.013164992 Billion total entry num = 0.013164992 Billion write speed all = 0.024125165 MB/sec (with seq num) write speed all = 0.022845013 MB/sec (without seq num) 2024/11/23-07:43:34.560096 140025196639808 [compaction_job.cc:1948] [default] [JOB 22043] Generated table #250928: 171335 keys, 7934267 bytes, temperature: kWarm 2024/11/23-07:43:34.560155 140025196639808 EVENT_LOG_v1 {"time_micros": 1732319014560125, "cf_name": "default", "job": 22043, "event": "table_file_creation", "file_number": 250928, "file_size": 7934267, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3742466073, "largest_seqno": 3776952302, "table_properties": {"data_size": 6831568, "index_size": 331456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4797380, "raw_average_key_size": 28, "raw_value_size": 20217530, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 171335, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732319014, "oldest_key_time": 0, "file_creation_time": 1732319014, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250928, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:43:34.571327 140025196639808 [compaction_job.cc:703] job-22043: subcompact[0], size: 0.527422 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22043/att-00 [250914,250915,250916,250917,250918,250919,250920,250921,250922,250923,250924,250925,250926,250927,250928] 2024/11/23-07:43:34.686161 140025196639808 (Original Log Time 2024/11/23-07:42:40.290537) EVENT_LOG_v1 {"time_micros": 1732318960290498, "job": 22043, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [250903], "files_L3": [250813, 250845, 250846, 250847, 250848, 250849, 250850, 250851, 250852, 250853, 250854, 250855, 250856, 250857], "score": 0.993527, "input_data_size": 552825900, "oldest_snapshot_seqno": 3865805356} 2024/11/23-07:43:34.686164 140025196639808 (Original Log Time 2024/11/23-07:43:34.684668) [compaction_job.cc:2022] [default] [JOB 22043] Compacted 1@2 + 14@3 files to L3 => 527422184 bytes 2024/11/23-07:43:34.686166 140025196639808 (Original Log Time 2024/11/23-07:43:34.686095) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 10.2 rd, 9.7 wr, level 3, files in(1, 14) out(15 +0 blob) MB in(27.1, 500.1 +0.0 blob) out(503.0 +0.0 blob), read-write-amplify(38.0) write-amplify(18.6) OK, records in: 13225012, records dropped: 60020 output_compression: Snappy 2024/11/23-07:43:34.686172 140025196639808 (Original Log Time 2024/11/23-07:43:34.686140) EVENT_LOG_v1 {"time_micros": 1732319014686119, "cf": "default", "job": 22043, "event": "compaction_finished", "compaction_time_micros": 54280798, "compaction_time_cpu_micros": 22552451, "output_level": 3, "num_output_files": 15, "total_output_size": 527422184, "num_input_records": 13225012, "num_output_records": 13164992, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 352414698, "file_range_sync_nanos": 0, "file_fsync_nanos": 13904, "file_prepare_write_nanos": 36586, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/23-07:43:34.804645 140025196639808 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732318960,"executesMs":54532,"compactionJobId":22043,"attempt":0,"compactionInputRawBytes":1912586644,"compactionInputZipBytes":552825900,"compactionOutputRawBytes":0,"compactionOutputZipBytes":527422184,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-0504ce3d2d36e476c989a7abef72130b-c4df630246a49f8c-00"}}, response = 2024/11/23-07:43:34.804719 140025196639808 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-22043/att-00: olev 3, work 54.532 s, result 4.941 ms, install 1.532 ms, input{raw 1.781 GiB zip 527.216 MiB}