2024/11/23-00:22:09.891567 140025163068992 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00: fork to child time = 0.028534 sec 2024/11/23-00:22:09.893881 140025163068992 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00 2024/11/23-00:22:09.905624 140025163068992 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00 2024/11/23-00:22:09.907800 140025163068992 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-00:22:09.908357 140025163068992 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-00:22:09.965324 140025163068992 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-15093/att-00/MANIFEST-206347 2024/11/23-00:22:09.999288 140025163068992 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-15093/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 236066, last_sequence is 3671958383, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 235914 2024/11/23-00:22:09.999301 140025163068992 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-00:22:09.999435 140025163068992 [version_set.cc:5721] Creating manifest 236066 2024/11/23-00:22:10.033822 140025163068992 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-00:22:10.054579 140025163068992 [compaction_job.cc:2437] [default] [JOB 15093] Compacting 1@2 + 12@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-00:22:10.054608 140025163068992 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 11311 Base level 2, inputs: [236064(25MB)], [235998(35MB) 235999(35MB) 236000(34MB) 236001(36MB) 236002(35MB) 236003(35MB) 236004(34MB) 236005(36MB) 236006(35MB) 236007(6939KB) 235964(35MB) 235965(17MB)] 2024/11/23-00:22:10.062747 140025163068992 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-00:22:10.062817 140025163068992 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-00:22:10.062846 140025163068992 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-00:22:10.062854 140025163068992 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-00:22:10.062868 140025163068992 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-00:22:10.532856 140025163068992 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236067.sst) /dev/shm/ToplingTemp/Topling-348668-ap3z4O 2024/11/23-00:22:11.657356 140025163068992 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236067.sst): first pass time = 1.12's, 119.364'MB/sec 2024/11/23-00:22:13.555540 140025163068992 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236067.sst): old prealloc_size = 147639500, real_size = 32903648 2024/11/23-00:22:13.556240 140025163068992 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236067.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926068 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 116.93 avg-zval = 33.38 2024/11/23-00:22:13.557062 140025163068992 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236067.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:22:14.899531 140025163068992 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236067.sst): second pass time = 1.90's, 57.102'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.34's, 27.530'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2598.411'MB/sec (index lex order gen) rebuild zvType time = 1.18's, 0.195'MB/sec write SST data time = 0.16's, 232.915'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 37216.903'MB/sec zip my value time = 1.90's, unzip length = 0.108'GB zip my value throughput = 57.102'MB/sec zip pipeline throughput = 57.102'MB/sec entries = 926068 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.93 avg-zval = 33.38 usrkeys = 926068 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.93 avg-zval = 33.38 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926068 bytes = 3241248 } seq expand size = 6482476 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5132 value = 3.5033 dict = 1.00 all = 3.6275 } Zip/UnZip{ index = 0.0951 value = 0.2854 dict = 1.00 all = 0.2757 } ---------------------------- total value len = 0.114770 GB avg = 0.124 KB (by entry num) total key len = 0.025930 GB avg = 0.028 KB total ukey len = 0.018521 GB avg = 0.020 KB total ukey num = 0.000926068 Billion total entry num = 0.000926068 Billion write speed all = 0.002521880 MB/sec (with seq num) write speed all = 0.002389091 MB/sec (without seq num) 2024/11/23-00:22:14.932514 140025163068992 [compaction_job.cc:1948] [default] [JOB 15093] Generated table #236067: 926068 keys, 36999814 bytes, temperature: kWarm 2024/11/23-00:22:14.932600 140025163068992 EVENT_LOG_v1 {"time_micros": 1732292534932547, "cf_name": "default", "job": 15093, "event": "table_file_creation", "file_number": 236067, "file_size": 36999814, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3534352946, "largest_seqno": 3671890400, "table_properties": {"data_size": 30910224, "index_size": 1761728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25929904, "raw_average_key_size": 28, "raw_value_size": 108287868, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926068, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732292530, "oldest_key_time": 0, "file_creation_time": 1732292530, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236067, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:22:14.969720 140025163068992 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236068.sst) /dev/shm/ToplingTemp/Topling-348668-kmhtnR 2024/11/23-00:22:16.274626 140025163068992 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236068.sst): first pass time = 1.30's, 102.859'MB/sec 2024/11/23-00:22:18.059904 140025163068992 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236068.sst): old prealloc_size = 147639500, real_size = 33048656 2024/11/23-00:22:18.060475 140025163068992 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236068.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928395 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.57 avg-zval = 33.45 2024/11/23-00:22:18.061232 140025163068992 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236068.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:22:19.015215 140025163068992 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236068.sst): second pass time = 1.78's, 60.645'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 38.890'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3199.310'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.281'MB/sec write SST data time = 0.13's, 290.601'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 36750.332'MB/sec zip my value time = 1.78's, unzip length = 0.108'GB zip my value throughput = 60.645'MB/sec zip pipeline throughput = 60.645'MB/sec entries = 928395 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.57 avg-zval = 33.45 usrkeys = 928395 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.57 avg-zval = 33.45 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928395 bytes = 3249392 } seq expand size = 6498765 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5148 value = 3.4854 dict = 1.00 all = 3.6128 } Zip/UnZip{ index = 0.0951 value = 0.2869 dict = 1.00 all = 0.2768 } ---------------------------- total value len = 0.229492 GB avg = 0.124 KB (by entry num) total key len = 0.051925 GB avg = 0.028 KB total ukey len = 0.037089 GB avg = 0.020 KB total ukey num = 0.001854463 Billion total entry num = 0.001854463 Billion write speed all = 0.005043682 MB/sec (with seq num) write speed all = 0.004777790 MB/sec (without seq num) 2024/11/23-00:22:19.048428 140025163068992 [compaction_job.cc:1948] [default] [JOB 15093] Generated table #236068: 928395 keys, 37151154 bytes, temperature: kWarm 2024/11/23-00:22:19.048491 140025163068992 EVENT_LOG_v1 {"time_micros": 1732292539048459, "cf_name": "default", "job": 15093, "event": "table_file_creation", "file_number": 236068, "file_size": 37151154, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 3671891021, "table_properties": {"data_size": 31050496, "index_size": 1765888, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25995060, "raw_average_key_size": 28, "raw_value_size": 108222794, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928395, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732292534, "oldest_key_time": 0, "file_creation_time": 1732292534, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236068, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:22:19.089066 140025163068992 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236069.sst) /dev/shm/ToplingTemp/Topling-348668-FXYggQ 2024/11/23-00:22:20.040666 140025163068992 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236069.sst): first pass time = 0.95's, 141.052'MB/sec 2024/11/23-00:22:21.853628 140025163068992 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236069.sst): old prealloc_size = 147639500, real_size = 32664864 2024/11/23-00:22:21.854192 140025163068992 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236069.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925191 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0307 GB avg-val = 117.07 avg-zval = 33.15 2024/11/23-00:22:21.854999 140025163068992 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236069.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:22:22.768729 140025163068992 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236069.sst): second pass time = 1.81's, 59.777'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.295'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3218.192'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.295'MB/sec write SST data time = 0.13's, 283.056'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 29238.920'MB/sec zip my value time = 1.81's, unzip length = 0.108'GB zip my value throughput = 59.777'MB/sec zip pipeline throughput = 124.199'MB/sec entries = 925191 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.07 avg-zval = 33.15 usrkeys = 925191 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.07 avg-zval = 33.15 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925191 bytes = 3353840 } seq expand size = 6476337 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5135 value = 3.5312 dict = 1.00 all = 3.6399 } Zip/UnZip{ index = 0.0951 value = 0.2832 dict = 1.00 all = 0.2747 } ---------------------------- total value len = 0.344281 GB avg = 0.124 KB (by entry num) total key len = 0.077830 GB avg = 0.028 KB total ukey len = 0.055593 GB avg = 0.020 KB total ukey num = 0.002779654 Billion total entry num = 0.002779654 Billion write speed all = 0.007564758 MB/sec (with seq num) write speed all = 0.007166239 MB/sec (without seq num) 2024/11/23-00:22:22.804838 140025163068992 [compaction_job.cc:1948] [default] [JOB 15093] Generated table #236069: 925191 keys, 36873780 bytes, temperature: kWarm 2024/11/23-00:22:22.804893 140025163068992 EVENT_LOG_v1 {"time_micros": 1732292542804864, "cf_name": "default", "job": 15093, "event": "table_file_creation", "file_number": 236069, "file_size": 36873780, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 3671891038, "table_properties": {"data_size": 30673360, "index_size": 1760000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25905348, "raw_average_key_size": 28, "raw_value_size": 108312450, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925191, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732292539, "oldest_key_time": 0, "file_creation_time": 1732292539, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236069, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:22:22.846068 140025163068992 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236070.sst) /dev/shm/ToplingTemp/Topling-348668-4G7qnS 2024/11/23-00:22:23.855893 140025163068992 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236070.sst): first pass time = 1.01's, 132.917'MB/sec 2024/11/23-00:22:25.640126 140025163068992 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236070.sst): old prealloc_size = 147639500, real_size = 34748624 2024/11/23-00:22:25.640696 140025163068992 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236070.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926823 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0327 GB avg-val = 116.81 avg-zval = 35.31 2024/11/23-00:22:25.641439 140025163068992 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236070.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:22:26.753738 140025163068992 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236070.sst): second pass time = 1.78's, 60.729'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.11's, 34.526'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3284.318'MB/sec (index lex order gen) rebuild zvType time = 0.88's, 0.264'MB/sec write SST data time = 0.24's, 162.957'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 28175.912'MB/sec zip my value time = 1.78's, unzip length = 0.108'GB zip my value throughput = 60.729'MB/sec zip pipeline throughput = 127.231'MB/sec entries = 926823 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.81 avg-zval = 35.31 usrkeys = 926823 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.43 avg-zval = 35.31 TagRS{ kinds = RS_Key0_TagN bytes = 144920 } TagArray{ size = 743826 bytes = 2696384 } seq expand size = 5206782 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0327 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.3785 value = 3.3078 dict = 1.00 all = 3.4909 } Zip/UnZip{ index = 0.0964 value = 0.3023 dict = 1.00 all = 0.2865 } ---------------------------- total value len = 0.457754 GB avg = 0.124 KB (by entry num) total key len = 0.103781 GB avg = 0.028 KB total ukey len = 0.074130 GB avg = 0.020 KB total ukey num = 0.003706477 Billion total entry num = 0.003706477 Billion write speed all = 0.010062703 MB/sec (with seq num) write speed all = 0.009531343 MB/sec (without seq num) 2024/11/23-00:22:26.788368 140025163068992 [compaction_job.cc:1948] [default] [JOB 15093] Generated table #236070: 926823 keys, 38447630 bytes, temperature: kWarm 2024/11/23-00:22:26.788430 140025163068992 EVENT_LOG_v1 {"time_micros": 1732292546788399, "cf_name": "default", "job": 15093, "event": "table_file_creation", "file_number": 236070, "file_size": 38447630, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3671890753, "table_properties": {"data_size": 32730688, "index_size": 1786048, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25951044, "raw_average_key_size": 28, "raw_value_size": 108266794, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926823, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732292542, "oldest_key_time": 0, "file_creation_time": 1732292542, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236070, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:22:26.828593 140025163068992 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236071.sst) /dev/shm/ToplingTemp/Topling-348668-9fCStR 2024/11/23-00:22:27.768950 140025163068992 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236071.sst): first pass time = 0.94's, 142.736'MB/sec 2024/11/23-00:22:29.287435 140025163068992 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236071.sst): old prealloc_size = 147639500, real_size = 32999152 2024/11/23-00:22:29.287968 140025163068992 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236071.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925889 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 116.96 avg-zval = 33.49 2024/11/23-00:22:29.288710 140025163068992 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236071.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:22:30.262366 140025163068992 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236071.sst): second pass time = 1.52's, 71.349'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.97's, 38.049'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3417.412'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.285'MB/sec write SST data time = 0.16's, 229.251'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34952.828'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 71.349'MB/sec zip pipeline throughput = 72.767'MB/sec entries = 925889 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.96 avg-zval = 33.49 usrkeys = 925889 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.96 avg-zval = 33.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925889 bytes = 3240624 } seq expand size = 6481223 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5150 value = 3.4926 dict = 1.00 all = 3.6183 } Zip/UnZip{ index = 0.0951 value = 0.2863 dict = 1.00 all = 0.2764 } ---------------------------- total value len = 0.572528 GB avg = 0.124 KB (by entry num) total key len = 0.129706 GB avg = 0.028 KB total ukey len = 0.092647 GB avg = 0.020 KB total ukey num = 0.004632366 Billion total entry num = 0.004632366 Billion write speed all = 0.012583233 MB/sec (with seq num) write speed all = 0.011919180 MB/sec (without seq num) 2024/11/23-00:22:30.295977 140025163068992 [compaction_job.cc:1948] [default] [JOB 15093] Generated table #236071: 925889 keys, 37094482 bytes, temperature: kWarm 2024/11/23-00:22:30.296041 140025163068992 EVENT_LOG_v1 {"time_micros": 1732292550296009, "cf_name": "default", "job": 15093, "event": "table_file_creation", "file_number": 236071, "file_size": 37094482, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 3671888381, "table_properties": {"data_size": 31006432, "index_size": 1761088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25924892, "raw_average_key_size": 28, "raw_value_size": 108292866, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925889, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732292546, "oldest_key_time": 0, "file_creation_time": 1732292546, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236071, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:22:30.333418 140025163068992 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236072.sst) /dev/shm/ToplingTemp/Topling-348668-qOxsGQ 2024/11/23-00:22:31.282844 140025163068992 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236072.sst): first pass time = 0.95's, 141.374'MB/sec 2024/11/23-00:22:32.799205 140025163068992 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236072.sst): old prealloc_size = 147639500, real_size = 33500080 2024/11/23-00:22:32.799760 140025163068992 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236072.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925987 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 116.95 avg-zval = 34.03 2024/11/23-00:22:32.800520 140025163068992 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236072.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:22:33.944892 140025163068992 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236072.sst): second pass time = 1.52's, 71.463'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.15's, 32.816'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3307.546'MB/sec (index lex order gen) rebuild zvType time = 0.99's, 0.234'MB/sec write SST data time = 0.16's, 239.966'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 27610.025'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 71.463'MB/sec zip pipeline throughput = 71.463'MB/sec entries = 925987 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.95 avg-zval = 34.03 usrkeys = 925987 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.95 avg-zval = 34.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925987 bytes = 3240976 } seq expand size = 6481909 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5134 value = 3.4370 dict = 1.00 all = 3.5700 } Zip/UnZip{ index = 0.0951 value = 0.2909 dict = 1.00 all = 0.2801 } ---------------------------- total value len = 0.687300 GB avg = 0.124 KB (by entry num) total key len = 0.155634 GB avg = 0.028 KB total ukey len = 0.111167 GB avg = 0.020 KB total ukey num = 0.005558353 Billion total entry num = 0.005558353 Billion write speed all = 0.015103414 MB/sec (with seq num) write speed all = 0.014306673 MB/sec (without seq num) 2024/11/23-00:22:33.978161 140025163068992 [compaction_job.cc:1948] [default] [JOB 15093] Generated table #236072: 925987 keys, 37596092 bytes, temperature: kWarm 2024/11/23-00:22:33.978228 140025163068992 EVENT_LOG_v1 {"time_micros": 1732292553978197, "cf_name": "default", "job": 15093, "event": "table_file_creation", "file_number": 236072, "file_size": 37596092, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 3671890767, "table_properties": {"data_size": 31506848, "index_size": 1761536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25927636, "raw_average_key_size": 28, "raw_value_size": 108290214, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925987, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732292550, "oldest_key_time": 0, "file_creation_time": 1732292550, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236072, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:22:34.015378 140025163068992 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236073.sst) /dev/shm/ToplingTemp/Topling-348668-wic1DO 2024/11/23-00:22:35.018775 140025163068992 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236073.sst): first pass time = 1.00's, 133.769'MB/sec 2024/11/23-00:22:36.597034 140025163068992 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236073.sst): old prealloc_size = 147639500, real_size = 32904288 2024/11/23-00:22:36.597559 140025163068992 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236073.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930169 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.29 avg-zval = 33.21 2024/11/23-00:22:36.598331 140025163068992 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236073.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:22:37.711971 140025163068992 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236073.sst): second pass time = 1.58's, 68.566'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.11's, 33.093'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3523.850'MB/sec (index lex order gen) rebuild zvType time = 0.98's, 0.236'MB/sec write SST data time = 0.13's, 283.015'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34633.667'MB/sec zip my value time = 1.58's, unzip length = 0.108'GB zip my value throughput = 68.566'MB/sec zip pipeline throughput = 68.566'MB/sec entries = 930169 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.29 avg-zval = 33.21 usrkeys = 930169 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.29 avg-zval = 33.21 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930169 bytes = 3139328 } seq expand size = 6511183 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4429 value = 3.5019 dict = 1.00 all = 3.6377 } Zip/UnZip{ index = 0.0958 value = 0.2856 dict = 1.00 all = 0.2749 } ---------------------------- total value len = 0.801985 GB avg = 0.124 KB (by entry num) total key len = 0.181679 GB avg = 0.028 KB total ukey len = 0.129770 GB avg = 0.020 KB total ukey num = 0.006488522 Billion total entry num = 0.006488522 Billion write speed all = 0.017623759 MB/sec (with seq num) write speed all = 0.016693749 MB/sec (without seq num) 2024/11/23-00:22:37.744151 140025163068992 [compaction_job.cc:1948] [default] [JOB 15093] Generated table #236073: 930169 keys, 36895884 bytes, temperature: kWarm 2024/11/23-00:22:37.744210 140025163068992 EVENT_LOG_v1 {"time_micros": 1732292557744180, "cf_name": "default", "job": 15093, "event": "table_file_creation", "file_number": 236073, "file_size": 36895884, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3604582272, "largest_seqno": 3671890795, "table_properties": {"data_size": 30890128, "index_size": 1781440, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26044732, "raw_average_key_size": 28, "raw_value_size": 108173062, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930169, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732292554, "oldest_key_time": 0, "file_creation_time": 1732292554, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236073, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:22:37.780157 140025163068992 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236074.sst) /dev/shm/ToplingTemp/Topling-348668-XvswVP 2024/11/23-00:22:38.800230 140025163068992 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236074.sst): first pass time = 1.02's, 131.582'MB/sec 2024/11/23-00:22:40.181605 140025163068992 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236074.sst): old prealloc_size = 147639500, real_size = 33402400 2024/11/23-00:22:40.182163 140025163068992 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236074.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932586 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0314 GB avg-val = 115.92 avg-zval = 33.66 2024/11/23-00:22:40.182910 140025163068992 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236074.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:22:41.132532 140025163068992 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236074.sst): second pass time = 1.38's, 78.300'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.453'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3293.719'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.283'MB/sec write SST data time = 0.13's, 293.199'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31570.989'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.300'MB/sec zip pipeline throughput = 78.300'MB/sec entries = 932586 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.92 avg-zval = 33.66 usrkeys = 932586 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.92 avg-zval = 33.66 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932586 bytes = 3264064 } seq expand size = 6528102 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5127 value = 3.4434 dict = 1.00 all = 3.5775 } Zip/UnZip{ index = 0.0951 value = 0.2904 dict = 1.00 all = 0.2795 } ---------------------------- total value len = 0.916618 GB avg = 0.124 KB (by entry num) total key len = 0.207791 GB avg = 0.028 KB total ukey len = 0.148422 GB avg = 0.020 KB total ukey num = 0.007421108 Billion total entry num = 0.007421108 Billion write speed all = 0.020144192 MB/sec (with seq num) write speed all = 0.019080578 MB/sec (without seq num) 2024/11/23-00:22:41.170698 140025163068992 [compaction_job.cc:1948] [default] [JOB 15093] Generated table #236074: 932586 keys, 37516922 bytes, temperature: kWarm 2024/11/23-00:22:41.170759 140025163068992 EVENT_LOG_v1 {"time_micros": 1732292561170728, "cf_name": "default", "job": 15093, "event": "table_file_creation", "file_number": 236074, "file_size": 37516922, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268533, "largest_seqno": 3671891154, "table_properties": {"data_size": 31394832, "index_size": 1774208, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26112408, "raw_average_key_size": 28, "raw_value_size": 108105400, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932586, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732292557, "oldest_key_time": 0, "file_creation_time": 1732292557, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236074, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:22:41.208004 140025163068992 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236075.sst) /dev/shm/ToplingTemp/Topling-348668-SVW2lR 2024/11/23-00:22:42.118746 140025163068992 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236075.sst): first pass time = 0.91's, 147.380'MB/sec 2024/11/23-00:22:43.467763 140025163068992 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236075.sst): old prealloc_size = 147639500, real_size = 32379936 2024/11/23-00:22:43.468313 140025163068992 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236075.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926416 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0304 GB avg-val = 116.88 avg-zval = 32.80 2024/11/23-00:22:43.470794 140025163068992 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236075.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:22:44.354111 140025163068992 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236075.sst): second pass time = 1.35's, 80.307'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 41.155'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3373.830'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.307'MB/sec write SST data time = 0.13's, 277.314'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30027.180'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.307'MB/sec zip pipeline throughput = 80.307'MB/sec entries = 926416 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.88 avg-zval = 32.80 usrkeys = 926416 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.88 avg-zval = 32.80 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926416 bytes = 3242464 } seq expand size = 6484912 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5114 value = 3.5635 dict = 1.00 all = 3.6795 } Zip/UnZip{ index = 0.0951 value = 0.2806 dict = 1.00 all = 0.2718 } ---------------------------- total value len = 1.031381 GB avg = 0.124 KB (by entry num) total key len = 0.233731 GB avg = 0.028 KB total ukey len = 0.166950 GB avg = 0.020 KB total ukey num = 0.008347524 Billion total entry num = 0.008347524 Billion write speed all = 0.022663623 MB/sec (with seq num) write speed all = 0.021467301 MB/sec (without seq num) 2024/11/23-00:22:44.386776 140025163068992 [compaction_job.cc:1948] [default] [JOB 15093] Generated table #236075: 926416 keys, 36477092 bytes, temperature: kWarm 2024/11/23-00:22:44.386848 140025163068992 EVENT_LOG_v1 {"time_micros": 1732292564386815, "cf_name": "default", "job": 15093, "event": "table_file_creation", "file_number": 236075, "file_size": 36477092, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 3671890841, "table_properties": {"data_size": 30385488, "index_size": 1762688, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25939648, "raw_average_key_size": 28, "raw_value_size": 108278152, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926416, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732292561, "oldest_key_time": 0, "file_creation_time": 1732292561, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236075, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:22:44.424951 140025163068992 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236076.sst) /dev/shm/ToplingTemp/Topling-348668-79aA1O 2024/11/23-00:22:45.400831 140025163068992 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236076.sst): first pass time = 0.98's, 137.541'MB/sec 2024/11/23-00:22:46.762071 140025163068992 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236076.sst): old prealloc_size = 147639500, real_size = 33370976 2024/11/23-00:22:46.762607 140025163068992 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236076.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925987 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.95 avg-zval = 33.89 2024/11/23-00:22:46.763408 140025163068992 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236076.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:22:47.706128 140025163068992 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236076.sst): second pass time = 1.36's, 79.589'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.688'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3419.963'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.284'MB/sec write SST data time = 0.13's, 290.750'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31435.253'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.589'MB/sec zip pipeline throughput = 79.589'MB/sec entries = 925987 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.95 avg-zval = 33.89 usrkeys = 925987 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.95 avg-zval = 33.89 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925987 bytes = 3240976 } seq expand size = 6481909 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5100 value = 3.4512 dict = 1.00 all = 3.5823 } Zip/UnZip{ index = 0.0951 value = 0.2898 dict = 1.00 all = 0.2792 } ---------------------------- total value len = 1.146153 GB avg = 0.124 KB (by entry num) total key len = 0.259658 GB avg = 0.028 KB total ukey len = 0.185470 GB avg = 0.020 KB total ukey num = 0.009273511 Billion total entry num = 0.009273511 Billion write speed all = 0.025182652 MB/sec (with seq num) write speed all = 0.023853702 MB/sec (without seq num) 2024/11/23-00:22:47.740883 140025163068992 [compaction_job.cc:1948] [default] [JOB 15093] Generated table #236076: 925987 keys, 37467000 bytes, temperature: kWarm 2024/11/23-00:22:47.740958 140025163068992 EVENT_LOG_v1 {"time_micros": 1732292567740916, "cf_name": "default", "job": 15093, "event": "table_file_creation", "file_number": 236076, "file_size": 37467000, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 3671890856, "table_properties": {"data_size": 31377168, "index_size": 1762112, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25927636, "raw_average_key_size": 28, "raw_value_size": 108290202, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925987, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732292564, "oldest_key_time": 0, "file_creation_time": 1732292564, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236076, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:22:47.785453 140025163068992 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236077.sst) /dev/shm/ToplingTemp/Topling-348668-ckr20O 2024/11/23-00:22:48.454883 140025163068992 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236077.sst): first pass time = 0.67's, 161.774'MB/sec 2024/11/23-00:22:49.554186 140025163068992 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236077.sst): old prealloc_size = 147639500, real_size = 26812080 2024/11/23-00:22:49.554658 140025163068992 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236077.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 753667 prefix = 4 raw-key = 0.0121 GB zip-key = 0.0014 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0872 GB zip-val = 0.0252 GB avg-val = 115.68 avg-zval = 33.41 2024/11/23-00:22:49.555334 140025163068992 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236077.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0019 GB, waited 0.000 sec, Key+Value = 0.108 GB 2024/11/23-00:22:50.320439 140025163068992 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00/236077.sst): second pass time = 1.10's, 79.363'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.77's, 39.333'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3221.918'MB/sec (index lex order gen) rebuild zvType time = 0.66's, 0.284'MB/sec write SST data time = 0.10's, 293.186'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.872'MB, throughput = 24915.510'MB/sec zip my value time = 1.10's, unzip length = 0.087'GB zip my value throughput = 79.363'MB/sec zip pipeline throughput = 79.363'MB/sec entries = 753667 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.68 avg-zval = 33.41 usrkeys = 753667 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.68 avg-zval = 33.41 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 753667 bytes = 2637856 } seq expand size = 5275669 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0151 GB value = 0.0872 GB dict = 0.87 MB all = 0.1083 GB } __ZipSize{ index = 0.0014 GB value = 0.0252 GB dict = 0.87 MB all = 0.0301 GB } UnZip/Zip{ index = 10.4411 value = 3.4626 dict = 1.00 all = 3.5930 } Zip/UnZip{ index = 0.0958 value = 0.2888 dict = 1.00 all = 0.2783 } ---------------------------- total value len = 1.238616 GB avg = 0.124 KB (by entry num) total key len = 0.280761 GB avg = 0.028 KB total ukey len = 0.200544 GB avg = 0.020 KB total ukey num = 0.010027178 Billion total entry num = 0.010027178 Billion write speed all = 0.027215694 MB/sec (with seq num) write speed all = 0.025778807 MB/sec (without seq num) 2024/11/23-00:22:50.348986 140025163068992 [compaction_job.cc:1948] [default] [JOB 15093] Generated table #236077: 753667 keys, 30138804 bytes, temperature: kWarm 2024/11/23-00:22:50.349050 140025163068992 EVENT_LOG_v1 {"time_micros": 1732292570349019, "cf_name": "default", "job": 15093, "event": "table_file_creation", "file_number": 236077, "file_size": 30138804, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 3671891075, "table_properties": {"data_size": 25179808, "index_size": 1443648, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 21102676, "raw_average_key_size": 28, "raw_value_size": 87186598, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 753667, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732292567, "oldest_key_time": 0, "file_creation_time": 1732292567, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236077, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:22:50.370398 140025163068992 [compaction_job.cc:703] job-15093: subcompact[0], size: 0.402659 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15093/att-00 [236067,236068,236069,236070,236071,236072,236073,236074,236075,236076,236077] 2024/11/23-00:22:50.818413 140025163068992 (Original Log Time 2024/11/23-00:22:10.054690) EVENT_LOG_v1 {"time_micros": 1732292530054630, "job": 15093, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [236064], "files_L3": [235998, 235999, 236000, 236001, 236002, 236003, 236004, 236005, 236006, 236007, 235964, 235965], "score": 0.999776, "input_data_size": 423821562, "oldest_snapshot_seqno": 3671958335} 2024/11/23-00:22:50.818416 140025163068992 (Original Log Time 2024/11/23-00:22:50.813324) [compaction_job.cc:2022] [default] [JOB 15093] Compacted 1@2 + 12@3 files to L3 => 402658654 bytes 2024/11/23-00:22:50.818417 140025163068992 (Original Log Time 2024/11/23-00:22:50.818347) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 11 0 0 0] max score 0.02, MB/sec: 10.5 rd, 10.0 wr, level 3, files in(1, 12) out(11 +0 blob) MB in(25.5, 378.7 +0.0 blob) out(384.0 +0.0 blob), read-write-amplify(30.9) write-amplify(15.0) OK, records in: 10099613, records dropped: 72435 output_compression: Snappy 2024/11/23-00:22:50.818424 140025163068992 (Original Log Time 2024/11/23-00:22:50.818392) EVENT_LOG_v1 {"time_micros": 1732292570818372, "cf": "default", "job": 15093, "event": "compaction_finished", "compaction_time_micros": 40315717, "compaction_time_cpu_micros": 18429329, "output_level": 3, "num_output_files": 11, "total_output_size": 402658654, "num_input_records": 10099613, "num_output_records": 10027178, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 258447591, "file_range_sync_nanos": 0, "file_fsync_nanos": 10049, "file_prepare_write_nanos": 44932, "lsm_state": [0, 0, 0, 11, 0, 0, 0]} 2024/11/23-00:22:50.947913 140025163068992 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732292529,"executesMs":40916,"compactionJobId":15093,"attempt":0,"compactionInputRawBytes":1460753154,"compactionInputZipBytes":423821562,"compactionOutputRawBytes":0,"compactionOutputZipBytes":402658654,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-77c760f07b9a646e33ad8ac5bac1d390-9fdc0b833f9de8e5-00"}}, response = 2024/11/23-00:22:50.947984 140025163068992 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-15093/att-00: olev 3, work 40.916 s, result 4.905 ms, install 5.120 ms, input{raw 1.360 GiB zip 404.188 MiB}