2024/11/22-23:10:27.932436 140024995214912 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00: fork to child time = 0.010344 sec 2024/11/22-23:10:27.933289 140024995214912 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00 2024/11/22-23:10:27.940562 140024995214912 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00 2024/11/22-23:10:27.941669 140024995214912 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-23:10:27.941943 140024995214912 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-23:10:27.987248 140024995214912 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-13642/att-00/MANIFEST-206347 2024/11/22-23:10:28.001225 140024995214912 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-13642/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 233102, last_sequence is 3633171249, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 232828 2024/11/22-23:10:28.001234 140024995214912 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-23:10:28.001367 140024995214912 [version_set.cc:5721] Creating manifest 233102 2024/11/22-23:10:28.036615 140024995214912 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-23:10:28.056130 140024995214912 [compaction_job.cc:2437] [default] [JOB 13642] Compacting 1@2 + 14@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/22-23:10:28.056153 140024995214912 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 10235 Base level 2, inputs: [233085(27MB)], [233017(34MB) 233018(35MB) 233019(35MB) 233020(35MB) 233021(35MB) 233022(34MB) 233023(35MB) 233024(33MB) 233025(34MB) 233026(37MB) 233027(35MB) 233028(36MB) 233029(35MB) 233090(37MB)] 2024/11/22-23:10:28.062259 140024995214912 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-23:10:28.062292 140024995214912 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-23:10:28.062313 140024995214912 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-23:10:28.062319 140024995214912 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-23:10:28.062330 140024995214912 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-23:10:28.608850 140024995214912 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233103.sst) /dev/shm/ToplingTemp/Topling-327769-0PJaaA 2024/11/22-23:10:29.305906 140024995214912 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233103.sst): first pass time = 0.70's, 192.567'MB/sec 2024/11/22-23:10:32.974239 140024995214912 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233103.sst): old prealloc_size = 147639500, real_size = 32443088 2024/11/22-23:10:32.974860 140024995214912 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233103.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925032 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0305 GB avg-val = 117.10 avg-zval = 32.92 2024/11/22-23:10:32.975737 140024995214912 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233103.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:10:34.156513 140024995214912 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233103.sst): second pass time = 3.67's, 29.535'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.18's, 30.805'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2917.802'MB/sec (index lex order gen) rebuild zvType time = 1.03's, 0.225'MB/sec write SST data time = 0.15's, 240.083'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 29699.049'MB/sec zip my value time = 3.67's, unzip length = 0.108'GB zip my value throughput = 29.535'MB/sec zip pipeline throughput = 117.941'MB/sec entries = 925032 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.10 avg-zval = 32.92 usrkeys = 925032 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.10 avg-zval = 32.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925032 bytes = 3122000 } seq expand size = 6475224 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5133 value = 3.5570 dict = 1.00 all = 3.6852 } Zip/UnZip{ index = 0.0951 value = 0.2811 dict = 1.00 all = 0.2714 } ---------------------------- total value len = 0.114792 GB avg = 0.124 KB (by entry num) total key len = 0.025901 GB avg = 0.028 KB total ukey len = 0.018501 GB avg = 0.020 KB total ukey num = 0.000925032 Billion total entry num = 0.000925032 Billion write speed all = 0.002732377 MB/sec (with seq num) write speed all = 0.002588658 MB/sec (without seq num) 2024/11/22-23:10:34.211062 140024995214912 [compaction_job.cc:1948] [default] [JOB 13642] Generated table #233103: 925032 keys, 36420650 bytes, temperature: kWarm 2024/11/22-23:10:34.211158 140024995214912 EVENT_LOG_v1 {"time_micros": 1732288234211095, "cf_name": "default", "job": 13642, "event": "table_file_creation", "file_number": 233103, "file_size": 36420650, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3513012344, "largest_seqno": 3633074379, "table_properties": {"data_size": 30451904, "index_size": 1759744, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25900896, "raw_average_key_size": 28, "raw_value_size": 108316856, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925032, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288228, "oldest_key_time": 0, "file_creation_time": 1732288228, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 233103, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:10:34.263601 140024995214912 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233104.sst) /dev/shm/ToplingTemp/Topling-327769-di9DPz 2024/11/22-23:10:35.724096 140024995214912 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233104.sst): first pass time = 1.46's, 91.901'MB/sec 2024/11/22-23:10:37.500019 140024995214912 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233104.sst): old prealloc_size = 147639500, real_size = 33291984 2024/11/22-23:10:37.500665 140024995214912 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233104.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924829 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 117.13 avg-zval = 33.85 2024/11/22-23:10:37.501600 140024995214912 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233104.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:10:39.098649 140024995214912 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233104.sst): second pass time = 1.78's, 61.023'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.60's, 23.386'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2818.424'MB/sec (index lex order gen) rebuild zvType time = 1.34's, 0.172'MB/sec write SST data time = 0.26's, 145.872'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35925.106'MB/sec zip my value time = 1.78's, unzip length = 0.108'GB zip my value throughput = 61.023'MB/sec zip pipeline throughput = 61.023'MB/sec entries = 924829 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.13 avg-zval = 33.85 usrkeys = 924829 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.13 avg-zval = 33.85 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924829 bytes = 3236912 } seq expand size = 6473803 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5129 value = 3.4607 dict = 1.00 all = 3.5902 } Zip/UnZip{ index = 0.0951 value = 0.2890 dict = 1.00 all = 0.2785 } ---------------------------- total value len = 0.229588 GB avg = 0.124 KB (by entry num) total key len = 0.051796 GB avg = 0.028 KB total ukey len = 0.036997 GB avg = 0.020 KB total ukey num = 0.001849861 Billion total entry num = 0.001849861 Billion write speed all = 0.005464201 MB/sec (with seq num) write speed all = 0.005176822 MB/sec (without seq num) 2024/11/22-23:10:39.137758 140024995214912 [compaction_job.cc:1948] [default] [JOB 13642] Generated table #233104: 924829 keys, 37384706 bytes, temperature: kWarm 2024/11/22-23:10:39.137842 140024995214912 EVENT_LOG_v1 {"time_micros": 1732288239137799, "cf_name": "default", "job": 13642, "event": "table_file_creation", "file_number": 233104, "file_size": 37384706, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038037, "largest_seqno": 3633074197, "table_properties": {"data_size": 31301184, "index_size": 1759424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25895212, "raw_average_key_size": 28, "raw_value_size": 108322534, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924829, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288234, "oldest_key_time": 0, "file_creation_time": 1732288234, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 233104, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:10:39.211591 140024995214912 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233105.sst) /dev/shm/ToplingTemp/Topling-327769-5xJBnz 2024/11/22-23:10:40.722461 140024995214912 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233105.sst): first pass time = 1.51's, 88.838'MB/sec 2024/11/22-23:10:42.706608 140024995214912 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233105.sst): old prealloc_size = 147639500, real_size = 32981296 2024/11/22-23:10:42.707245 140024995214912 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233105.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925131 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 117.08 avg-zval = 33.50 2024/11/22-23:10:42.708062 140024995214912 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233105.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:10:43.909052 140024995214912 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233105.sst): second pass time = 1.98's, 54.611'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.20's, 30.833'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2857.206'MB/sec (index lex order gen) rebuild zvType time = 1.04's, 0.222'MB/sec write SST data time = 0.16's, 234.099'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35735.203'MB/sec zip my value time = 1.98's, unzip length = 0.108'GB zip my value throughput = 54.611'MB/sec zip pipeline throughput = 56.863'MB/sec entries = 925131 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.08 avg-zval = 33.50 usrkeys = 925131 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.08 avg-zval = 33.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925131 bytes = 3237968 } seq expand size = 6475917 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5132 value = 3.4951 dict = 1.00 all = 3.6202 } Zip/UnZip{ index = 0.0951 value = 0.2861 dict = 1.00 all = 0.2762 } ---------------------------- total value len = 0.344378 GB avg = 0.124 KB (by entry num) total key len = 0.077700 GB avg = 0.028 KB total ukey len = 0.055500 GB avg = 0.020 KB total ukey num = 0.002774992 Billion total entry num = 0.002774992 Billion write speed all = 0.008195564 MB/sec (with seq num) write speed all = 0.007764504 MB/sec (without seq num) 2024/11/22-23:10:43.942855 140024995214912 [compaction_job.cc:1948] [default] [JOB 13642] Generated table #233105: 925131 keys, 37074576 bytes, temperature: kWarm 2024/11/22-23:10:43.942927 140024995214912 EVENT_LOG_v1 {"time_micros": 1732288243942885, "cf_name": "default", "job": 13642, "event": "table_file_creation", "file_number": 233105, "file_size": 37074576, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3491168223, "largest_seqno": 3633074514, "table_properties": {"data_size": 30989920, "index_size": 1759936, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25903668, "raw_average_key_size": 28, "raw_value_size": 108314118, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925131, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288239, "oldest_key_time": 0, "file_creation_time": 1732288239, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 233105, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:10:43.982622 140024995214912 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233106.sst) /dev/shm/ToplingTemp/Topling-327769-hrXM5z 2024/11/22-23:10:44.818271 140024995214912 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233106.sst): first pass time = 0.84's, 160.622'MB/sec 2024/11/22-23:10:46.218364 140024995214912 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233106.sst): old prealloc_size = 147639500, real_size = 33554480 2024/11/22-23:10:46.218954 140024995214912 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233106.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924924 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 117.11 avg-zval = 34.10 2024/11/22-23:10:46.219688 140024995214912 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233106.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:10:47.263274 140024995214912 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233106.sst): second pass time = 1.40's, 77.403'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.04's, 36.030'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3129.240'MB/sec (index lex order gen) rebuild zvType time = 0.91's, 0.253'MB/sec write SST data time = 0.13's, 289.192'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33463.939'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.403'MB/sec zip pipeline throughput = 77.403'MB/sec entries = 924924 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.11 avg-zval = 34.10 usrkeys = 924924 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.11 avg-zval = 34.10 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924924 bytes = 3237248 } seq expand size = 6474468 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.3810 value = 3.4342 dict = 1.00 all = 3.5651 } Zip/UnZip{ index = 0.0963 value = 0.2912 dict = 1.00 all = 0.2805 } ---------------------------- total value len = 0.459173 GB avg = 0.124 KB (by entry num) total key len = 0.103598 GB avg = 0.028 KB total ukey len = 0.073998 GB avg = 0.020 KB total ukey num = 0.003699916 Billion total entry num = 0.003699916 Billion write speed all = 0.010926697 MB/sec (with seq num) write speed all = 0.010352000 MB/sec (without seq num) 2024/11/22-23:10:47.304456 140024995214912 [compaction_job.cc:1948] [default] [JOB 13642] Generated table #233106: 924924 keys, 37647502 bytes, temperature: kWarm 2024/11/22-23:10:47.304519 140024995214912 EVENT_LOG_v1 {"time_micros": 1732288247304488, "cf_name": "default", "job": 13642, "event": "table_file_creation", "file_number": 233106, "file_size": 37647502, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038075, "largest_seqno": 3633074161, "table_properties": {"data_size": 31541088, "index_size": 1781952, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25897872, "raw_average_key_size": 28, "raw_value_size": 108319876, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924924, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288243, "oldest_key_time": 0, "file_creation_time": 1732288243, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 233106, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:10:47.342974 140024995214912 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233107.sst) /dev/shm/ToplingTemp/Topling-327769-9sKvQy 2024/11/22-23:10:48.225026 140024995214912 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233107.sst): first pass time = 0.88's, 152.173'MB/sec 2024/11/22-23:10:49.615596 140024995214912 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233107.sst): old prealloc_size = 147639500, real_size = 33239072 2024/11/22-23:10:49.616174 140024995214912 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233107.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926512 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 116.86 avg-zval = 33.72 2024/11/22-23:10:49.616930 140024995214912 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233107.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:10:50.661146 140024995214912 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233107.sst): second pass time = 1.39's, 77.951'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.05's, 35.600'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3164.207'MB/sec (index lex order gen) rebuild zvType time = 0.91's, 0.253'MB/sec write SST data time = 0.13's, 284.637'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36201.106'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 77.951'MB/sec zip pipeline throughput = 77.951'MB/sec entries = 926512 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.86 avg-zval = 33.72 usrkeys = 926512 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.86 avg-zval = 33.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926512 bytes = 3126992 } seq expand size = 6485584 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5110 value = 3.4654 dict = 1.00 all = 3.6060 } Zip/UnZip{ index = 0.0951 value = 0.2886 dict = 1.00 all = 0.2773 } ---------------------------- total value len = 0.573934 GB avg = 0.124 KB (by entry num) total key len = 0.129540 GB avg = 0.028 KB total ukey len = 0.092529 GB avg = 0.020 KB total ukey num = 0.004626428 Billion total entry num = 0.004626428 Billion write speed all = 0.013657679 MB/sec (with seq num) write speed all = 0.012939116 MB/sec (without seq num) 2024/11/22-23:10:50.696191 140024995214912 [compaction_job.cc:1948] [default] [JOB 13642] Generated table #233107: 926512 keys, 37220662 bytes, temperature: kWarm 2024/11/22-23:10:50.696254 140024995214912 EVENT_LOG_v1 {"time_micros": 1732288250696222, "cf_name": "default", "job": 13642, "event": "table_file_creation", "file_number": 233107, "file_size": 37220662, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3523718385, "largest_seqno": 3633073037, "table_properties": {"data_size": 31244304, "index_size": 1762944, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25942336, "raw_average_key_size": 28, "raw_value_size": 108275432, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926512, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288247, "oldest_key_time": 0, "file_creation_time": 1732288247, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 233107, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:10:50.735801 140024995214912 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233108.sst) /dev/shm/ToplingTemp/Topling-327769-OQJO8v 2024/11/22-23:10:51.522631 140024995214912 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233108.sst): first pass time = 0.79's, 170.592'MB/sec 2024/11/22-23:10:52.881479 140024995214912 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233108.sst): old prealloc_size = 147639500, real_size = 32372432 2024/11/22-23:10:52.882073 140024995214912 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233108.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924436 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0304 GB avg-val = 117.19 avg-zval = 32.85 2024/11/22-23:10:52.882837 140024995214912 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233108.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:10:53.872579 140024995214912 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233108.sst): second pass time = 1.36's, 79.813'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.99's, 36.792'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3095.381'MB/sec (index lex order gen) rebuild zvType time = 0.86's, 0.269'MB/sec write SST data time = 0.13's, 280.294'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35002.330'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.813'MB/sec zip pipeline throughput = 79.813'MB/sec entries = 924436 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.19 avg-zval = 32.85 usrkeys = 924436 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.19 avg-zval = 32.85 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924436 bytes = 3235536 } seq expand size = 6471052 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.4438 value = 3.5670 dict = 1.00 all = 3.6808 } Zip/UnZip{ index = 0.0958 value = 0.2803 dict = 1.00 all = 0.2717 } ---------------------------- total value len = 0.688738 GB avg = 0.124 KB (by entry num) total key len = 0.155424 GB avg = 0.028 KB total ukey len = 0.111017 GB avg = 0.020 KB total ukey num = 0.005550864 Billion total entry num = 0.005550864 Billion write speed all = 0.016388078 MB/sec (with seq num) write speed all = 0.015525988 MB/sec (without seq num) 2024/11/22-23:10:53.908156 140024995214912 [compaction_job.cc:1948] [default] [JOB 13642] Generated table #233108: 924436 keys, 36464104 bytes, temperature: kWarm 2024/11/22-23:10:53.908222 140024995214912 EVENT_LOG_v1 {"time_micros": 1732288253908188, "cf_name": "default", "job": 13642, "event": "table_file_creation", "file_number": 233108, "file_size": 36464104, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038106, "largest_seqno": 3633074589, "table_properties": {"data_size": 30370816, "index_size": 1770304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25884208, "raw_average_key_size": 28, "raw_value_size": 108333576, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924436, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288250, "oldest_key_time": 0, "file_creation_time": 1732288250, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 233108, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:10:53.949488 140024995214912 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233109.sst) /dev/shm/ToplingTemp/Topling-327769-SKuQUy 2024/11/22-23:10:54.854381 140024995214912 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233109.sst): first pass time = 0.90's, 148.331'MB/sec 2024/11/22-23:10:56.234195 140024995214912 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233109.sst): old prealloc_size = 147639500, real_size = 32295856 2024/11/22-23:10:56.234886 140024995214912 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233109.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929477 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0303 GB avg-val = 116.40 avg-zval = 32.59 2024/11/22-23:10:56.235669 140024995214912 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233109.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:10:57.224530 140024995214912 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233109.sst): second pass time = 1.38's, 78.451'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.99's, 36.757'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2637.989'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.280'MB/sec write SST data time = 0.16's, 229.395'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 28208.099'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.451'MB/sec zip pipeline throughput = 78.451'MB/sec entries = 929477 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.40 avg-zval = 32.59 usrkeys = 929477 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.40 avg-zval = 32.59 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929477 bytes = 3253184 } seq expand size = 6506339 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5152 value = 3.5712 dict = 1.00 all = 3.6871 } Zip/UnZip{ index = 0.0951 value = 0.2800 dict = 1.00 all = 0.2712 } ---------------------------- total value len = 0.803437 GB avg = 0.124 KB (by entry num) total key len = 0.181450 GB avg = 0.028 KB total ukey len = 0.129607 GB avg = 0.020 KB total ukey num = 0.006480341 Billion total entry num = 0.006480341 Billion write speed all = 0.019118771 MB/sec (with seq num) write speed all = 0.018112392 MB/sec (without seq num) 2024/11/22-23:10:57.264812 140024995214912 [compaction_job.cc:1948] [default] [JOB 13642] Generated table #233109: 929477 keys, 36401564 bytes, temperature: kWarm 2024/11/22-23:10:57.264872 140024995214912 EVENT_LOG_v1 {"time_micros": 1732288257264841, "cf_name": "default", "job": 13642, "event": "table_file_creation", "file_number": 233109, "file_size": 36401564, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038136, "largest_seqno": 3633074313, "table_properties": {"data_size": 30295456, "index_size": 1767872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26025356, "raw_average_key_size": 28, "raw_value_size": 108192486, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929477, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288253, "oldest_key_time": 0, "file_creation_time": 1732288253, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 233109, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:10:57.301909 140024995214912 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233110.sst) /dev/shm/ToplingTemp/Topling-327769-Kqa3Jx 2024/11/22-23:10:58.171002 140024995214912 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233110.sst): first pass time = 0.87's, 154.441'MB/sec 2024/11/22-23:10:59.482883 140024995214912 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233110.sst): old prealloc_size = 147639500, real_size = 31013888 2024/11/22-23:10:59.483443 140024995214912 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233110.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924008 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0290 GB avg-val = 117.26 avg-zval = 31.40 2024/11/22-23:10:59.484218 140024995214912 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233110.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:11:00.344872 140024995214912 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233110.sst): second pass time = 1.31's, 82.631'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 40.724'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3299.240'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.316'MB/sec write SST data time = 0.13's, 268.905'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 33555.686'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.631'MB/sec zip pipeline throughput = 82.631'MB/sec entries = 924008 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.26 avg-zval = 31.40 usrkeys = 924008 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.26 avg-zval = 31.40 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924008 bytes = 3234048 } seq expand size = 6468056 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0290 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.4450 value = 3.7343 dict = 1.00 all = 3.8235 } Zip/UnZip{ index = 0.0957 value = 0.2678 dict = 1.00 all = 0.2615 } ---------------------------- total value len = 0.918251 GB avg = 0.124 KB (by entry num) total key len = 0.207322 GB avg = 0.028 KB total ukey len = 0.148087 GB avg = 0.020 KB total ukey num = 0.007404349 Billion total entry num = 0.007404349 Billion write speed all = 0.021848463 MB/sec (with seq num) write speed all = 0.020698658 MB/sec (without seq num) 2024/11/22-23:11:00.379536 140024995214912 [compaction_job.cc:1948] [default] [JOB 13642] Generated table #233110: 924008 keys, 35103182 bytes, temperature: kWarm 2024/11/22-23:11:00.379597 140024995214912 EVENT_LOG_v1 {"time_micros": 1732288260379567, "cf_name": "default", "job": 13642, "event": "table_file_creation", "file_number": 233110, "file_size": 35103182, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3437662288, "largest_seqno": 3633071481, "table_properties": {"data_size": 29013424, "index_size": 1769280, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25872224, "raw_average_key_size": 28, "raw_value_size": 108345620, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924008, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288257, "oldest_key_time": 0, "file_creation_time": 1732288257, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 233110, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:11:00.421387 140024995214912 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233111.sst) /dev/shm/ToplingTemp/Topling-327769-ADXhHy 2024/11/22-23:11:01.344019 140024995214912 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233111.sst): first pass time = 0.92's, 145.480'MB/sec 2024/11/22-23:11:02.701576 140024995214912 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233111.sst): old prealloc_size = 147639500, real_size = 31646848 2024/11/22-23:11:02.702181 140024995214912 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233111.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926012 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0297 GB avg-val = 116.94 avg-zval = 32.02 2024/11/22-23:11:02.703857 140024995214912 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233111.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:11:03.573417 140024995214912 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233111.sst): second pass time = 1.36's, 79.811'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 41.135'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3034.997'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.314'MB/sec write SST data time = 0.13's, 269.846'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35520.494'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.811'MB/sec zip pipeline throughput = 79.811'MB/sec entries = 926012 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.94 avg-zval = 32.02 usrkeys = 926012 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.94 avg-zval = 32.02 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926012 bytes = 3356816 } seq expand size = 6482084 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.08 MB all = 0.0359 GB } UnZip/Zip{ index = 10.5133 value = 3.6518 dict = 1.00 all = 3.7426 } Zip/UnZip{ index = 0.0951 value = 0.2738 dict = 1.00 all = 0.2672 } ---------------------------- total value len = 1.033022 GB avg = 0.124 KB (by entry num) total key len = 0.233250 GB avg = 0.028 KB total ukey len = 0.166607 GB avg = 0.020 KB total ukey num = 0.008330361 Billion total entry num = 0.008330361 Billion write speed all = 0.024578044 MB/sec (with seq num) write speed all = 0.023284522 MB/sec (without seq num) 2024/11/22-23:11:03.609575 140024995214912 [compaction_job.cc:1948] [default] [JOB 13642] Generated table #233111: 926012 keys, 35862416 bytes, temperature: kWarm 2024/11/22-23:11:03.609639 140024995214912 EVENT_LOG_v1 {"time_micros": 1732288263609607, "cf_name": "default", "job": 13642, "event": "table_file_creation", "file_number": 233111, "file_size": 35862416, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868609, "largest_seqno": 3633074610, "table_properties": {"data_size": 29653552, "index_size": 1761600, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25928336, "raw_average_key_size": 28, "raw_value_size": 108289420, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926012, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288260, "oldest_key_time": 0, "file_creation_time": 1732288260, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 233111, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:11:03.650522 140024995214912 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233112.sst) /dev/shm/ToplingTemp/Topling-327769-SJck7z 2024/11/22-23:11:04.496239 140024995214912 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233112.sst): first pass time = 0.85's, 158.711'MB/sec 2024/11/22-23:11:05.922571 140024995214912 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233112.sst): old prealloc_size = 147639500, real_size = 37042112 2024/11/22-23:11:05.923129 140024995214912 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233112.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925228 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0351 GB avg-val = 117.06 avg-zval = 37.88 2024/11/22-23:11:05.923517 140024995214912 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233112.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:11:06.644789 140024995214912 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233112.sst): second pass time = 1.43's, 75.973'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.72's, 54.434'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3284.369'MB/sec (index lex order gen) rebuild zvType time = 0.59's, 0.391'MB/sec write SST data time = 0.13's, 303.172'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33779.514'MB/sec zip my value time = 1.43's, unzip length = 0.108'GB zip my value throughput = 75.973'MB/sec zip pipeline throughput = 75.973'MB/sec entries = 925228 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.06 avg-zval = 37.88 usrkeys = 925228 avg-key = 20.00 avg-zkey = 1.90 avg-val = 120.01 avg-zval = 37.88 TagRS{ kinds = RS_Key0_TagN bytes = 144680 } TagArray{ size = 389501 bytes = 1265888 } seq expand size = 2726507 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0351 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.5128 value = 3.0902 dict = 1.00 all = 3.4142 } Zip/UnZip{ index = 0.0951 value = 0.3236 dict = 1.00 all = 0.2929 } ---------------------------- total value len = 1.144060 GB avg = 0.124 KB (by entry num) total key len = 0.259156 GB avg = 0.028 KB total ukey len = 0.185112 GB avg = 0.020 KB total ukey num = 0.009255589 Billion total entry num = 0.009255589 Billion write speed all = 0.027234476 MB/sec (with seq num) write speed all = 0.025797372 MB/sec (without seq num) 2024/11/22-23:11:06.682247 140024995214912 [compaction_job.cc:1948] [default] [JOB 13642] Generated table #233112: 925228 keys, 39311635 bytes, temperature: kWarm 2024/11/22-23:11:06.682310 140024995214912 EVENT_LOG_v1 {"time_micros": 1732288266682278, "cf_name": "default", "job": 13642, "event": "table_file_creation", "file_number": 233112, "file_size": 39311635, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700833, "table_properties": {"data_size": 35050416, "index_size": 1760192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25906384, "raw_average_key_size": 28, "raw_value_size": 108311420, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925228, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288263, "oldest_key_time": 0, "file_creation_time": 1732288263, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 233112, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:11:06.717347 140024995214912 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233113.sst) /dev/shm/ToplingTemp/Topling-327769-1IIobz 2024/11/22-23:11:07.480706 140024995214912 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233113.sst): first pass time = 0.76's, 175.835'MB/sec 2024/11/22-23:11:09.050036 140024995214912 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233113.sst): old prealloc_size = 147639500, real_size = 36462880 2024/11/22-23:11:09.050561 140024995214912 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233113.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927051 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.78 avg-zval = 37.19 2024/11/22-23:11:09.050628 140024995214912 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233113.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:11:09.692920 140024995214912 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233113.sst): second pass time = 1.57's, 69.016'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 58.055'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3322.294'MB/sec (index lex order gen) rebuild zvType time = 0.52's, 0.450'MB/sec write SST data time = 0.13's, 294.044'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33625.335'MB/sec zip my value time = 1.57's, unzip length = 0.108'GB zip my value throughput = 69.016'MB/sec zip pipeline throughput = 69.016'MB/sec entries = 927051 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.78 avg-zval = 37.19 usrkeys = 927051 avg-key = 20.00 avg-zkey = 1.89 avg-val = 116.78 avg-zval = 37.19 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 7 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5820 value = 3.1399 dict = 1.00 all = 3.5963 } Zip/UnZip{ index = 0.0945 value = 0.3185 dict = 1.00 all = 0.2781 } ---------------------------- total value len = 1.252321 GB avg = 0.123 KB (by entry num) total key len = 0.285114 GB avg = 0.028 KB total ukey len = 0.203653 GB avg = 0.020 KB total ukey num = 0.010182640 Billion total entry num = 0.010182640 Billion write speed all = 0.029837692 MB/sec (with seq num) write speed all = 0.028256739 MB/sec (without seq num) 2024/11/22-23:11:09.728032 140024995214912 [compaction_job.cc:1948] [default] [JOB 13642] Generated table #233113: 927051 keys, 37320960 bytes, temperature: kWarm 2024/11/22-23:11:09.728097 140024995214912 EVENT_LOG_v1 {"time_micros": 1732288269728064, "cf_name": "default", "job": 13642, "event": "table_file_creation", "file_number": 233113, "file_size": 37320960, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34478800, "index_size": 1752128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25957428, "raw_average_key_size": 28, "raw_value_size": 108260418, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927051, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288266, "oldest_key_time": 0, "file_creation_time": 1732288266, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 233113, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:11:09.772957 140024995214912 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233114.sst) /dev/shm/ToplingTemp/Topling-327769-BKyciw 2024/11/22-23:11:10.531246 140024995214912 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233114.sst): first pass time = 0.76's, 177.012'MB/sec 2024/11/22-23:11:12.014585 140024995214912 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233114.sst): old prealloc_size = 147639500, real_size = 36804640 2024/11/22-23:11:12.015142 140024995214912 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233114.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925237 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0348 GB avg-val = 117.06 avg-zval = 37.61 2024/11/22-23:11:12.015403 140024995214912 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233114.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:11:12.712291 140024995214912 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233114.sst): second pass time = 1.48's, 73.080'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.70's, 55.170'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3306.361'MB/sec (index lex order gen) rebuild zvType time = 0.57's, 0.406'MB/sec write SST data time = 0.13's, 303.710'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 30386.683'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.080'MB/sec zip pipeline throughput = 73.080'MB/sec entries = 925237 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.06 avg-zval = 37.61 usrkeys = 925237 avg-key = 20.00 avg-zkey = 1.92 avg-val = 118.65 avg-zval = 37.61 TagRS{ kinds = RS_Key0_TagN bytes = 144680 } TagArray{ size = 210244 bytes = 683312 } seq expand size = 1471708 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0348 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.4430 value = 3.1123 dict = 1.00 all = 3.4869 } Zip/UnZip{ index = 0.0958 value = 0.3213 dict = 1.00 all = 0.2868 } ---------------------------- total value len = 1.362104 GB avg = 0.123 KB (by entry num) total key len = 0.311021 GB avg = 0.028 KB total ukey len = 0.222158 GB avg = 0.020 KB total ukey num = 0.011107877 Billion total entry num = 0.011107877 Billion write speed all = 0.032469178 MB/sec (with seq num) write speed all = 0.030744674 MB/sec (without seq num) 2024/11/22-23:11:12.752199 140024995214912 [compaction_job.cc:1948] [default] [JOB 13642] Generated table #233114: 925237 keys, 38491631 bytes, temperature: kWarm 2024/11/22-23:11:12.752262 140024995214912 EVENT_LOG_v1 {"time_micros": 1732288272752231, "cf_name": "default", "job": 13642, "event": "table_file_creation", "file_number": 233114, "file_size": 38491631, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3316967376, "table_properties": {"data_size": 34801168, "index_size": 1771968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25906636, "raw_average_key_size": 28, "raw_value_size": 108311230, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925237, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288269, "oldest_key_time": 0, "file_creation_time": 1732288269, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 233114, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:11:12.788279 140024995214912 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233115.sst) /dev/shm/ToplingTemp/Topling-327769-2JS26v 2024/11/22-23:11:13.569960 140024995214912 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233115.sst): first pass time = 0.78's, 171.713'MB/sec 2024/11/22-23:11:14.910817 140024995214912 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233115.sst): old prealloc_size = 147639500, real_size = 32101600 2024/11/22-23:11:14.911353 140024995214912 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233115.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925409 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0301 GB avg-val = 117.04 avg-zval = 32.52 2024/11/22-23:11:14.912147 140024995214912 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233115.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:11:15.796300 140024995214912 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233115.sst): second pass time = 1.34's, 80.816'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 41.008'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3445.203'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.306'MB/sec write SST data time = 0.13's, 282.496'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34602.562'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.816'MB/sec zip pipeline throughput = 80.816'MB/sec entries = 925409 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.04 avg-zval = 32.52 usrkeys = 925409 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.04 avg-zval = 32.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925409 bytes = 3354624 } seq expand size = 6477863 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.4431 value = 3.5985 dict = 1.00 all = 3.6963 } Zip/UnZip{ index = 0.0958 value = 0.2779 dict = 1.00 all = 0.2705 } ---------------------------- total value len = 1.476888 GB avg = 0.123 KB (by entry num) total key len = 0.336932 GB avg = 0.028 KB total ukey len = 0.240666 GB avg = 0.020 KB total ukey num = 0.012033286 Billion total entry num = 0.012033286 Billion write speed all = 0.035197457 MB/sec (with seq num) write speed all = 0.033329394 MB/sec (without seq num) 2024/11/22-23:11:15.831810 140024995214912 [compaction_job.cc:1948] [default] [JOB 13642] Generated table #233115: 925409 keys, 36311310 bytes, temperature: kWarm 2024/11/22-23:11:15.831872 140024995214912 EVENT_LOG_v1 {"time_micros": 1732288275831841, "cf_name": "default", "job": 13642, "event": "table_file_creation", "file_number": 233115, "file_size": 36311310, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3316967377, "largest_seqno": 3633072685, "table_properties": {"data_size": 30097744, "index_size": 1772288, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25911452, "raw_average_key_size": 28, "raw_value_size": 108306382, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925409, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288272, "oldest_key_time": 0, "file_creation_time": 1732288272, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 233115, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:11:15.870042 140024995214912 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233116.sst) /dev/shm/ToplingTemp/Topling-327769-1tBCuw 2024/11/22-23:11:16.700431 140024995214912 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233116.sst): first pass time = 0.83's, 161.641'MB/sec 2024/11/22-23:11:18.052844 140024995214912 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233116.sst): old prealloc_size = 147639500, real_size = 34823568 2024/11/22-23:11:18.053378 140024995214912 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233116.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929479 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0328 GB avg-val = 116.40 avg-zval = 35.30 2024/11/22-23:11:18.054170 140024995214912 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233116.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:11:18.956344 140024995214912 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233116.sst): second pass time = 1.35's, 80.038'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 43.217'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3467.800'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.300'MB/sec write SST data time = 0.13's, 301.875'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34006.616'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.038'MB/sec zip pipeline throughput = 80.038'MB/sec entries = 929479 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.40 avg-zval = 35.30 usrkeys = 929479 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.40 avg-zval = 35.30 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929479 bytes = 3369376 } seq expand size = 6506353 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0328 GB dict = 1.08 MB all = 0.0390 GB } UnZip/Zip{ index = 10.4434 value = 3.2974 dict = 1.00 all = 3.4375 } Zip/UnZip{ index = 0.0958 value = 0.3033 dict = 1.00 all = 0.2909 } ---------------------------- total value len = 1.591587 GB avg = 0.123 KB (by entry num) total key len = 0.362957 GB avg = 0.028 KB total ukey len = 0.259255 GB avg = 0.020 KB total ukey num = 0.012962765 Billion total entry num = 0.012962765 Billion write speed all = 0.037925906 MB/sec (with seq num) write speed all = 0.035913674 MB/sec (without seq num) 2024/11/22-23:11:18.991738 140024995214912 [compaction_job.cc:1948] [default] [JOB 13642] Generated table #233116: 929479 keys, 39045752 bytes, temperature: kWarm 2024/11/22-23:11:18.991808 140024995214912 EVENT_LOG_v1 {"time_micros": 1732288278991769, "cf_name": "default", "job": 13642, "event": "table_file_creation", "file_number": 233116, "file_size": 39045752, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701386, "largest_seqno": 3633073962, "table_properties": {"data_size": 32811008, "index_size": 1780032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26025412, "raw_average_key_size": 28, "raw_value_size": 108192458, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929479, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288275, "oldest_key_time": 0, "file_creation_time": 1732288275, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 233116, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:11:19.042849 140024995214912 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233117.sst) /dev/shm/ToplingTemp/Topling-327769-342lLz 2024/11/22-23:11:19.134711 140024995214912 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233117.sst): first pass time = 0.09's, 228.704'MB/sec 2024/11/22-23:11:19.317855 140024995214912 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233117.sst): old prealloc_size = 147639500, real_size = 5863888 2024/11/22-23:11:19.317990 140024995214912 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233117.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 146529 prefix = 4 raw-key = 0.0023 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0169 GB zip-val = 0.0055 GB avg-val = 115.32 avg-zval = 37.84 2024/11/22-23:11:19.318138 140024995214912 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233117.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.021 GB 2024/11/22-23:11:19.456928 140024995214912 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00/233117.sst): second pass time = 0.18's, 92.558'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.14's, 46.490'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2540.562'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.309'MB/sec write SST data time = 0.02's, 315.813'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.178'MB, throughput = 5969.851'MB/sec zip my value time = 0.18's, unzip length = 0.017'GB zip my value throughput = 92.558'MB/sec zip pipeline throughput = 92.558'MB/sec entries = 146529 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.32 avg-zval = 37.84 usrkeys = 146529 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.32 avg-zval = 37.84 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 146529 bytes = 457920 } seq expand size = 1025703 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0029 GB value = 0.0169 GB dict = 0.18 MB all = 0.0210 GB } __ZipSize{ index = 0.0003 GB value = 0.0055 GB dict = 0.18 MB all = 0.0065 GB } UnZip/Zip{ index = 10.4022 value = 3.0471 dict = 1.00 all = 3.2483 } Zip/UnZip{ index = 0.0961 value = 0.3282 dict = 1.00 all = 0.3079 } ---------------------------- total value len = 1.609510 GB avg = 0.123 KB (by entry num) total key len = 0.367060 GB avg = 0.028 KB total ukey len = 0.262186 GB avg = 0.020 KB total ukey num = 0.013109294 Billion total entry num = 0.013109294 Billion write speed all = 0.038352918 MB/sec (with seq num) write speed all = 0.036317959 MB/sec (without seq num) 2024/11/22-23:11:19.469400 140024995214912 [compaction_job.cc:1948] [default] [JOB 13642] Generated table #233117: 146529 keys, 6464883 bytes, temperature: kWarm 2024/11/22-23:11:19.469465 140024995214912 EVENT_LOG_v1 {"time_micros": 1732288279469427, "cf_name": "default", "job": 13642, "event": "table_file_creation", "file_number": 233117, "file_size": 6464883, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3603604769, "largest_seqno": 3632510422, "table_properties": {"data_size": 5545344, "index_size": 281728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4102812, "raw_average_key_size": 28, "raw_value_size": 16897086, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 146529, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288279, "oldest_key_time": 0, "file_creation_time": 1732288279, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 233117, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:11:19.480585 140024995214912 [compaction_job.cc:703] job-13642: subcompact[0], size: 0.526526 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13642/att-00 [233103,233104,233105,233106,233107,233108,233109,233110,233111,233112,233113,233114,233115,233116,233117] 2024/11/22-23:11:19.575198 140024995214912 (Original Log Time 2024/11/22-23:10:28.056211) EVENT_LOG_v1 {"time_micros": 1732288228056169, "job": 13642, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [233085], "files_L3": [233017, 233018, 233019, 233020, 233021, 233022, 233023, 233024, 233025, 233026, 233027, 233028, 233029, 233090], "score": 0.998756, "input_data_size": 550091242, "oldest_snapshot_seqno": 3633171202} 2024/11/22-23:11:19.575202 140024995214912 (Original Log Time 2024/11/22-23:11:19.573741) [compaction_job.cc:2022] [default] [JOB 13642] Compacted 1@2 + 14@3 files to L3 => 526525533 bytes 2024/11/22-23:11:19.575203 140024995214912 (Original Log Time 2024/11/22-23:11:19.575131) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 10.7 rd, 10.2 wr, level 3, files in(1, 14) out(15 +0 blob) MB in(27.1, 497.5 +0.0 blob) out(502.1 +0.0 blob), read-write-amplify(37.9) write-amplify(18.5) OK, records in: 13197452, records dropped: 88158 output_compression: Snappy 2024/11/22-23:11:19.575211 140024995214912 (Original Log Time 2024/11/22-23:11:19.575177) EVENT_LOG_v1 {"time_micros": 1732288279575155, "cf": "default", "job": 13642, "event": "compaction_finished", "compaction_time_micros": 51424382, "compaction_time_cpu_micros": 22104811, "output_level": 3, "num_output_files": 15, "total_output_size": 526525533, "num_input_records": 13197452, "num_output_records": 13109294, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 371655207, "file_range_sync_nanos": 0, "file_fsync_nanos": 15131, "file_prepare_write_nanos": 35665, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/22-23:11:19.707536 140024995214912 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732288227,"executesMs":51636,"compactionJobId":13642,"attempt":0,"compactionInputRawBytes":1912567580,"compactionInputZipBytes":550091242,"compactionOutputRawBytes":0,"compactionOutputZipBytes":526525533,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-5ec3520a4bbc3fd163339727f2caea5c-6b5a5deec20073c6-00"}}, response = 2024/11/22-23:11:19.707607 140024995214912 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-13642/att-00: olev 3, work 51.636 s, result 4.879 ms, install 1.494 ms, input{raw 1.781 GiB zip 524.608 MiB}