2024/11/23-11:30:15.951755 139957525694016 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00: fork to child time = 0.005963 sec 2024/11/23-11:30:15.952316 139957525694016 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00 2024/11/23-11:30:15.959658 139957525694016 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00 2024/11/23-11:30:15.960576 139957525694016 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-11:30:15.960749 139957525694016 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-11:30:16.012897 139957525694016 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-25225/att-00/MANIFEST-206347 2024/11/23-11:30:16.038420 139957525694016 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-25225/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 257818, last_sequence is 3953002530, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 257618 2024/11/23-11:30:16.038466 139957525694016 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-11:30:16.038559 139957525694016 [version_set.cc:5721] Creating manifest 257818 2024/11/23-11:30:16.073293 139957525694016 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-11:30:16.092405 139957525694016 [compaction_job.cc:2437] [default] [JOB 25225] Compacting 1@2 + 12@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-11:30:16.092416 139957525694016 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 18903 Base level 2, inputs: [257767(27MB)], [257689(36MB) 257690(34MB) 257691(34MB) 257692(35MB) 257693(35MB) 257694(36MB) 257695(35MB) 257696(36MB) 257697(35MB) 257719(35MB) 257720(35MB) 257805(35MB)] 2024/11/23-11:30:16.100621 139957525694016 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-11:30:16.100652 139957525694016 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-11:30:16.100666 139957525694016 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-11:30:16.100674 139957525694016 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-11:30:16.100680 139957525694016 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-11:30:16.661279 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257819.sst) /dev/shm/ToplingTemp/Topling-37170-nLk0sO 2024/11/23-11:30:17.218936 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257819.sst): first pass time = 0.56's, 240.695'MB/sec 2024/11/23-11:30:18.887050 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257819.sst): old prealloc_size = 147639500, real_size = 33515648 2024/11/23-11:30:18.887586 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257819.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933181 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0315 GB avg-val = 115.83 avg-zval = 33.75 2024/11/23-11:30:18.888401 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257819.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:30:19.737088 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257819.sst): second pass time = 1.67's, 64.852'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.272'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3501.620'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 294.796'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35803.799'MB/sec zip my value time = 1.67's, unzip length = 0.108'GB zip my value throughput = 64.852'MB/sec zip pipeline throughput = 129.824'MB/sec entries = 933181 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.83 avg-zval = 33.75 usrkeys = 933181 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.83 avg-zval = 33.75 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933181 bytes = 3266144 } seq expand size = 6532267 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.4448 value = 3.4319 dict = 1.00 all = 3.5665 } Zip/UnZip{ index = 0.0957 value = 0.2914 dict = 1.00 all = 0.2804 } ---------------------------- total value len = 8.963241 GB avg = 0.168 KB (by entry num) total key len = 1.466950 GB avg = 0.028 KB total ukey len = 1.016542 GB avg = 0.015 KB total ukey num = 0.066483616 Billion total entry num = 0.053291345 Billion write speed all = 1.120619476 MB/sec (with seq num) write speed all = 1.074814509 MB/sec (without seq num) 2024/11/23-11:30:19.809821 139957525694016 [compaction_job.cc:1948] [default] [JOB 25225] Generated table #257819: 933181 keys, 37632508 bytes, temperature: kWarm 2024/11/23-11:30:19.809945 139957525694016 EVENT_LOG_v1 {"time_micros": 1732332619809857, "cf_name": "default", "job": 25225, "event": "table_file_creation", "file_number": 257819, "file_size": 37632508, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3711387651, "largest_seqno": 3952701353, "table_properties": {"data_size": 31495280, "index_size": 1786880, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26129068, "raw_average_key_size": 28, "raw_value_size": 108088706, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933181, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732332616, "oldest_key_time": 0, "file_creation_time": 1732332616, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 257819, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:30:19.850971 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257820.sst) /dev/shm/ToplingTemp/Topling-37170-oB0CPM 2024/11/23-11:30:20.746650 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257820.sst): first pass time = 0.90's, 149.859'MB/sec 2024/11/23-11:30:22.251835 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257820.sst): old prealloc_size = 147639500, real_size = 32487648 2024/11/23-11:30:22.252391 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257820.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931368 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0305 GB avg-val = 116.11 avg-zval = 32.73 2024/11/23-11:30:22.253160 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257820.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:30:23.105546 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257820.sst): second pass time = 1.50's, 71.883'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 42.735'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3313.314'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.13's, 288.534'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33654.492'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 71.883'MB/sec zip pipeline throughput = 115.162'MB/sec entries = 931368 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.11 avg-zval = 32.73 usrkeys = 931368 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.11 avg-zval = 32.73 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931368 bytes = 3143376 } seq expand size = 6519576 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5115 value = 3.5476 dict = 1.00 all = 3.6790 } Zip/UnZip{ index = 0.0951 value = 0.2819 dict = 1.00 all = 0.2718 } ---------------------------- total value len = 9.077900 GB avg = 0.167 KB (by entry num) total key len = 1.493028 GB avg = 0.028 KB total ukey len = 1.035169 GB avg = 0.015 KB total ukey num = 0.067414984 Billion total entry num = 0.054222713 Billion write speed all = 1.135329417 MB/sec (with seq num) write speed all = 1.088740781 MB/sec (without seq num) 2024/11/23-11:30:23.149405 139957525694016 [compaction_job.cc:1948] [default] [JOB 25225] Generated table #257820: 931368 keys, 36482072 bytes, temperature: kWarm 2024/11/23-11:30:23.149470 139957525694016 EVENT_LOG_v1 {"time_micros": 1732332623149436, "cf_name": "default", "job": 25225, "event": "table_file_creation", "file_number": 257820, "file_size": 36482072, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3855348712, "largest_seqno": 3952701019, "table_properties": {"data_size": 30482512, "index_size": 1772096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26078304, "raw_average_key_size": 28, "raw_value_size": 108139540, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931368, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732332619, "oldest_key_time": 0, "file_creation_time": 1732332619, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 257820, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:30:23.187257 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257821.sst) /dev/shm/ToplingTemp/Topling-37170-Z6uVuP 2024/11/23-11:30:24.063120 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257821.sst): first pass time = 0.88's, 153.247'MB/sec 2024/11/23-11:30:25.488627 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257821.sst): old prealloc_size = 147639500, real_size = 31718720 2024/11/23-11:30:25.489201 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257821.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934914 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1080 GB zip-val = 0.0297 GB avg-val = 115.56 avg-zval = 31.76 2024/11/23-11:30:25.489934 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257821.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:30:26.330727 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257821.sst): second pass time = 1.42's, 75.834'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.424'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3306.028'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.326'MB/sec write SST data time = 0.12's, 288.340'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36684.750'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 75.834'MB/sec zip pipeline throughput = 75.834'MB/sec entries = 934914 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.56 avg-zval = 31.76 usrkeys = 934914 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.56 avg-zval = 31.76 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934914 bytes = 3155344 } seq expand size = 6544398 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.4418 value = 3.6384 dict = 1.00 all = 3.7571 } Zip/UnZip{ index = 0.0958 value = 0.2748 dict = 1.00 all = 0.2662 } ---------------------------- total value len = 9.192485 GB avg = 0.167 KB (by entry num) total key len = 1.519206 GB avg = 0.028 KB total ukey len = 1.053867 GB avg = 0.015 KB total ukey num = 0.068349898 Billion total entry num = 0.055157627 Billion write speed all = 1.150049078 MB/sec (with seq num) write speed all = 1.102673566 MB/sec (without seq num) 2024/11/23-11:30:26.374214 139957525694016 [compaction_job.cc:1948] [default] [JOB 25225] Generated table #257821: 934914 keys, 35724068 bytes, temperature: kWarm 2024/11/23-11:30:26.374282 139957525694016 EVENT_LOG_v1 {"time_micros": 1732332626374247, "cf_name": "default", "job": 25225, "event": "table_file_creation", "file_number": 257821, "file_size": 35724068, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3845180379, "largest_seqno": 3952701033, "table_properties": {"data_size": 29694064, "index_size": 1790720, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26177592, "raw_average_key_size": 28, "raw_value_size": 108040280, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934914, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732332623, "oldest_key_time": 0, "file_creation_time": 1732332623, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 257821, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:30:26.408614 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257822.sst) /dev/shm/ToplingTemp/Topling-37170-OUf3CP 2024/11/23-11:30:27.362467 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257822.sst): first pass time = 0.95's, 140.717'MB/sec 2024/11/23-11:30:28.769060 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257822.sst): old prealloc_size = 147639500, real_size = 32904320 2024/11/23-11:30:28.769609 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257822.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928468 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.56 avg-zval = 33.29 2024/11/23-11:30:28.770396 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257822.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:30:29.628469 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257822.sst): second pass time = 1.41's, 76.987'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.928'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3345.518'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.316'MB/sec write SST data time = 0.13's, 294.796'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 26086.625'MB/sec zip my value time = 1.41's, unzip length = 0.108'GB zip my value throughput = 76.987'MB/sec zip pipeline throughput = 76.987'MB/sec entries = 928468 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.56 avg-zval = 33.29 usrkeys = 928468 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.56 avg-zval = 33.29 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928468 bytes = 3133600 } seq expand size = 6499276 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5156 value = 3.5016 dict = 1.00 all = 3.6382 } Zip/UnZip{ index = 0.0951 value = 0.2856 dict = 1.00 all = 0.2749 } ---------------------------- total value len = 9.307205 GB avg = 0.166 KB (by entry num) total key len = 1.545203 GB avg = 0.028 KB total ukey len = 1.072437 GB avg = 0.015 KB total ukey num = 0.069278366 Billion total entry num = 0.056086095 Billion write speed all = 1.164744627 MB/sec (with seq num) write speed all = 1.116588693 MB/sec (without seq num) 2024/11/23-11:30:29.674516 139957525694016 [compaction_job.cc:1948] [default] [JOB 25225] Generated table #257822: 928468 keys, 36891572 bytes, temperature: kWarm 2024/11/23-11:30:29.674587 139957525694016 EVENT_LOG_v1 {"time_micros": 1732332629674553, "cf_name": "default", "job": 25225, "event": "table_file_creation", "file_number": 257822, "file_size": 36891572, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3847905997, "largest_seqno": 3952701054, "table_properties": {"data_size": 30906160, "index_size": 1765888, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25997104, "raw_average_key_size": 28, "raw_value_size": 108220692, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928468, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732332626, "oldest_key_time": 0, "file_creation_time": 1732332626, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 257822, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:30:29.723213 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257823.sst) /dev/shm/ToplingTemp/Topling-37170-jRIlfN 2024/11/23-11:30:30.770113 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257823.sst): first pass time = 1.05's, 128.210'MB/sec 2024/11/23-11:30:32.172862 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257823.sst): old prealloc_size = 147639500, real_size = 33111008 2024/11/23-11:30:32.173436 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257823.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933531 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0311 GB avg-val = 115.77 avg-zval = 33.32 2024/11/23-11:30:32.174216 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257823.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:30:33.033219 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257823.sst): second pass time = 1.40's, 77.088'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.136'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3225.317'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 296.110'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34945.758'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.088'MB/sec zip pipeline throughput = 77.088'MB/sec entries = 933531 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.77 avg-zval = 33.32 usrkeys = 933531 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.77 avg-zval = 33.32 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933531 bytes = 3150688 } seq expand size = 6534717 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5112 value = 3.4751 dict = 1.00 all = 3.6166 } Zip/UnZip{ index = 0.0951 value = 0.2878 dict = 1.00 all = 0.2765 } ---------------------------- total value len = 9.421818 GB avg = 0.165 KB (by entry num) total key len = 1.571342 GB avg = 0.028 KB total ukey len = 1.091107 GB avg = 0.016 KB total ukey num = 0.070211897 Billion total entry num = 0.057019626 Billion write speed all = 1.179420030 MB/sec (with seq num) write speed all = 1.130480443 MB/sec (without seq num) 2024/11/23-11:30:33.079394 139957525694016 [compaction_job.cc:1948] [default] [JOB 25225] Generated table #257823: 933531 keys, 37112024 bytes, temperature: kWarm 2024/11/23-11:30:33.079459 139957525694016 EVENT_LOG_v1 {"time_micros": 1732332633079426, "cf_name": "default", "job": 25225, "event": "table_file_creation", "file_number": 257823, "file_size": 37112024, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3880929219, "largest_seqno": 3952701697, "table_properties": {"data_size": 31101200, "index_size": 1776256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26138868, "raw_average_key_size": 28, "raw_value_size": 108078862, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933531, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732332629, "oldest_key_time": 0, "file_creation_time": 1732332629, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 257823, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:30:33.113971 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257824.sst) /dev/shm/ToplingTemp/Topling-37170-jJQONP 2024/11/23-11:30:34.018420 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257824.sst): first pass time = 0.90's, 148.413'MB/sec 2024/11/23-11:30:35.524303 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257824.sst): old prealloc_size = 147639500, real_size = 36031888 2024/11/23-11:30:35.524876 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257824.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931139 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0340 GB avg-val = 116.14 avg-zval = 36.53 2024/11/23-11:30:35.525394 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257824.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:30:36.296923 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257824.sst): second pass time = 1.51's, 71.853'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.77's, 50.339'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3234.519'MB/sec (index lex order gen) rebuild zvType time = 0.65's, 0.360'MB/sec write SST data time = 0.13's, 309.127'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35173.730'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.853'MB/sec zip pipeline throughput = 71.853'MB/sec entries = 931139 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.14 avg-zval = 36.53 usrkeys = 931139 avg-key = 20.00 avg-zkey = 1.91 avg-val = 119.87 avg-zval = 36.53 TagRS{ kinds = RS_Key0_TagN bytes = 145600 } TagArray{ size = 495870 bytes = 1859520 } seq expand size = 3471090 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0340 GB dict = 1.08 MB all = 0.0389 GB } UnZip/Zip{ index = 10.4451 value = 3.1793 dict = 1.00 all = 3.4510 } Zip/UnZip{ index = 0.0957 value = 0.3145 dict = 1.00 all = 0.2898 } ---------------------------- total value len = 9.533435 GB avg = 0.165 KB (by entry num) total key len = 1.597414 GB avg = 0.028 KB total ukey len = 1.109730 GB avg = 0.016 KB total ukey num = 0.071143036 Billion total entry num = 0.057950765 Billion write speed all = 1.193774218 MB/sec (with seq num) write speed all = 1.144052851 MB/sec (without seq num) 2024/11/23-11:30:36.344258 139957525694016 [compaction_job.cc:1948] [default] [JOB 25225] Generated table #257824: 931139 keys, 38892141 bytes, temperature: kWarm 2024/11/23-11:30:36.344323 139957525694016 EVENT_LOG_v1 {"time_micros": 1732332636344288, "cf_name": "default", "job": 25225, "event": "table_file_creation", "file_number": 257824, "file_size": 38892141, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3952701102, "table_properties": {"data_size": 34016000, "index_size": 1782912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26071892, "raw_average_key_size": 28, "raw_value_size": 108145922, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931139, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732332633, "oldest_key_time": 0, "file_creation_time": 1732332633, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 257824, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:30:36.382152 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257825.sst) /dev/shm/ToplingTemp/Topling-37170-82AnzQ 2024/11/23-11:30:37.281045 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257825.sst): first pass time = 0.90's, 149.320'MB/sec 2024/11/23-11:30:38.908507 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257825.sst): old prealloc_size = 147639500, real_size = 36420512 2024/11/23-11:30:38.909084 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257825.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927101 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0344 GB avg-val = 116.77 avg-zval = 37.14 2024/11/23-11:30:38.909124 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257825.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:30:39.609089 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257825.sst): second pass time = 1.63's, 66.552'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.70's, 53.212'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3153.120'MB/sec (index lex order gen) rebuild zvType time = 0.58's, 0.402'MB/sec write SST data time = 0.12's, 303.586'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36465.631'MB/sec zip my value time = 1.63's, unzip length = 0.108'GB zip my value throughput = 66.552'MB/sec zip pipeline throughput = 66.552'MB/sec entries = 927101 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.77 avg-zval = 37.14 usrkeys = 927101 avg-key = 20.00 avg-zkey = 1.89 avg-val = 116.77 avg-zval = 37.14 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5822 value = 3.1437 dict = 1.00 all = 3.6004 } Zip/UnZip{ index = 0.0945 value = 0.3181 dict = 1.00 all = 0.2777 } ---------------------------- total value len = 9.641694 GB avg = 0.164 KB (by entry num) total key len = 1.623373 GB avg = 0.028 KB total ukey len = 1.128272 GB avg = 0.016 KB total ukey num = 0.072070137 Billion total entry num = 0.058877866 Billion write speed all = 1.207739941 MB/sec (with seq num) write speed all = 1.157241065 MB/sec (without seq num) 2024/11/23-11:30:39.657662 139957525694016 [compaction_job.cc:1948] [default] [JOB 25225] Generated table #257825: 927101 keys, 37278322 bytes, temperature: kWarm 2024/11/23-11:30:39.657731 139957525694016 EVENT_LOG_v1 {"time_micros": 1732332639657694, "cf_name": "default", "job": 25225, "event": "table_file_creation", "file_number": 257825, "file_size": 37278322, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34436368, "index_size": 1752192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25958828, "raw_average_key_size": 28, "raw_value_size": 108258950, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927101, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732332636, "oldest_key_time": 0, "file_creation_time": 1732332636, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 257825, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:30:39.694717 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257826.sst) /dev/shm/ToplingTemp/Topling-37170-W0kGIP 2024/11/23-11:30:40.736511 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257826.sst): first pass time = 1.04's, 128.838'MB/sec 2024/11/23-11:30:42.192501 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257826.sst): old prealloc_size = 147639500, real_size = 35013104 2024/11/23-11:30:42.193035 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257826.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927918 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1082 GB zip-val = 0.0330 GB avg-val = 116.64 avg-zval = 35.59 2024/11/23-11:30:42.193684 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257826.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:30:43.001671 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257826.sst): second pass time = 1.45's, 74.402'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 47.573'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3416.667'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.340'MB/sec write SST data time = 0.13's, 303.596'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 29780.376'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.402'MB/sec zip pipeline throughput = 74.402'MB/sec entries = 927918 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.64 avg-zval = 35.59 usrkeys = 927918 avg-key = 20.00 avg-zkey = 1.89 avg-val = 121.63 avg-zval = 35.59 TagRS{ kinds = RS_Key0_TagN bytes = 145080 } TagArray{ size = 660797 bytes = 2478000 } seq expand size = 4625579 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0330 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.5818 value = 3.2772 dict = 1.00 all = 3.4868 } Zip/UnZip{ index = 0.0945 value = 0.3051 dict = 1.00 all = 0.2868 } ---------------------------- total value len = 9.754556 GB avg = 0.163 KB (by entry num) total key len = 1.649354 GB avg = 0.028 KB total ukey len = 1.146830 GB avg = 0.016 KB total ukey num = 0.072998055 Billion total entry num = 0.059805784 Billion write speed all = 1.222180961 MB/sec (with seq num) write speed all = 1.170904871 MB/sec (without seq num) 2024/11/23-11:30:43.048955 139957525694016 [compaction_job.cc:1948] [default] [JOB 25225] Generated table #257826: 927918 keys, 38493672 bytes, temperature: kWarm 2024/11/23-11:30:43.049034 139957525694016 EVENT_LOG_v1 {"time_micros": 1732332643049000, "cf_name": "default", "job": 25225, "event": "table_file_creation", "file_number": 257826, "file_size": 38493672, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3952701381, "table_properties": {"data_size": 33027168, "index_size": 1753792, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25981704, "raw_average_key_size": 28, "raw_value_size": 108236156, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927918, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732332639, "oldest_key_time": 0, "file_creation_time": 1732332639, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 257826, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:30:43.083109 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257827.sst) /dev/shm/ToplingTemp/Topling-37170-gLb66N 2024/11/23-11:30:44.131197 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257827.sst): first pass time = 1.05's, 128.064'MB/sec 2024/11/23-11:30:45.556779 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257827.sst): old prealloc_size = 147639500, real_size = 33114256 2024/11/23-11:30:45.557311 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257827.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930225 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.29 avg-zval = 33.45 2024/11/23-11:30:45.558185 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257827.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:30:46.419859 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257827.sst): second pass time = 1.42's, 75.921'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.262'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3470.471'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.12's, 300.240'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33579.092'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 75.921'MB/sec zip pipeline throughput = 75.921'MB/sec entries = 930225 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.29 avg-zval = 33.45 usrkeys = 930225 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.29 avg-zval = 33.45 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930225 bytes = 3372080 } seq expand size = 6511575 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5134 value = 3.4768 dict = 1.00 all = 3.5947 } Zip/UnZip{ index = 0.0951 value = 0.2876 dict = 1.00 all = 0.2782 } ---------------------------- total value len = 9.869239 GB avg = 0.162 KB (by entry num) total key len = 1.675401 GB avg = 0.028 KB total ukey len = 1.165435 GB avg = 0.016 KB total ukey num = 0.073928280 Billion total entry num = 0.060736009 Billion write speed all = 1.236810139 MB/sec (with seq num) write speed all = 1.184755565 MB/sec (without seq num) 2024/11/23-11:30:46.465039 139957525694016 [compaction_job.cc:1948] [default] [JOB 25225] Generated table #257827: 930225 keys, 37338194 bytes, temperature: kWarm 2024/11/23-11:30:46.465108 139957525694016 EVENT_LOG_v1 {"time_micros": 1732332646465073, "cf_name": "default", "job": 25225, "event": "table_file_creation", "file_number": 257827, "file_size": 37338194, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3631595348, "largest_seqno": 3952701748, "table_properties": {"data_size": 31111936, "index_size": 1769600, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26046300, "raw_average_key_size": 28, "raw_value_size": 108171522, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930225, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732332643, "oldest_key_time": 0, "file_creation_time": 1732332643, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 257827, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:30:46.506102 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257828.sst) /dev/shm/ToplingTemp/Topling-37170-tdRPQO 2024/11/23-11:30:47.387602 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257828.sst): first pass time = 0.88's, 152.270'MB/sec 2024/11/23-11:30:48.849640 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257828.sst): old prealloc_size = 147639500, real_size = 32882960 2024/11/23-11:30:48.850165 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257828.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930195 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.29 avg-zval = 33.20 2024/11/23-11:30:48.851627 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257828.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:30:49.709243 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257828.sst): second pass time = 1.46's, 74.086'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.034'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3495.471'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 295.567'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 28437.773'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 74.086'MB/sec zip pipeline throughput = 74.086'MB/sec entries = 930195 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.29 avg-zval = 33.20 usrkeys = 930195 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.29 avg-zval = 33.20 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930195 bytes = 3255696 } seq expand size = 6511365 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5119 value = 3.5029 dict = 1.00 all = 3.6284 } Zip/UnZip{ index = 0.0951 value = 0.2855 dict = 1.00 all = 0.2756 } ---------------------------- total value len = 9.983923 GB avg = 0.162 KB (by entry num) total key len = 1.701446 GB avg = 0.028 KB total ukey len = 1.184039 GB avg = 0.016 KB total ukey num = 0.074858475 Billion total entry num = 0.061666204 Billion write speed all = 1.251445846 MB/sec (with seq num) write speed all = 1.198612655 MB/sec (without seq num) 2024/11/23-11:30:49.755706 139957525694016 [compaction_job.cc:1948] [default] [JOB 25225] Generated table #257828: 930195 keys, 36990938 bytes, temperature: kWarm 2024/11/23-11:30:49.755773 139957525694016 EVENT_LOG_v1 {"time_micros": 1732332649755738, "cf_name": "default", "job": 25225, "event": "table_file_creation", "file_number": 257828, "file_size": 36990938, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3736072251, "largest_seqno": 3952701481, "table_properties": {"data_size": 30880448, "index_size": 1769792, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26045460, "raw_average_key_size": 28, "raw_value_size": 108172354, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930195, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732332646, "oldest_key_time": 0, "file_creation_time": 1732332646, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 257828, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:30:49.794967 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257829.sst) /dev/shm/ToplingTemp/Topling-37170-tsFufN 2024/11/23-11:30:50.822257 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257829.sst): first pass time = 1.03's, 130.663'MB/sec 2024/11/23-11:30:52.266425 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257829.sst): old prealloc_size = 147639500, real_size = 32841808 2024/11/23-11:30:52.266956 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257829.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930745 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0308 GB avg-val = 116.20 avg-zval = 33.12 2024/11/23-11:30:52.268578 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257829.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:30:53.131426 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257829.sst): second pass time = 1.44's, 74.956'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.854'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3521.225'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.316'MB/sec write SST data time = 0.13's, 290.126'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34740.369'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 74.956'MB/sec zip pipeline throughput = 74.956'MB/sec entries = 930745 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.20 avg-zval = 33.12 usrkeys = 930745 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.20 avg-zval = 33.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930745 bytes = 3373968 } seq expand size = 6515215 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.4449 value = 3.5085 dict = 1.00 all = 3.6209 } Zip/UnZip{ index = 0.0957 value = 0.2850 dict = 1.00 all = 0.2762 } ---------------------------- total value len = 10.098595 GB avg = 0.161 KB (by entry num) total key len = 1.727507 GB avg = 0.028 KB total ukey len = 1.202654 GB avg = 0.016 KB total ukey num = 0.075789220 Billion total entry num = 0.062596949 Billion write speed all = 1.266053658 MB/sec (with seq num) write speed all = 1.212442690 MB/sec (without seq num) 2024/11/23-11:30:53.179755 139957525694016 [compaction_job.cc:1948] [default] [JOB 25225] Generated table #257829: 930745 keys, 37067360 bytes, temperature: kWarm 2024/11/23-11:30:53.179820 139957525694016 EVENT_LOG_v1 {"time_micros": 1732332653179786, "cf_name": "default", "job": 25225, "event": "table_file_creation", "file_number": 257829, "file_size": 37067360, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 3952700790, "table_properties": {"data_size": 30826752, "index_size": 1782208, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26060860, "raw_average_key_size": 28, "raw_value_size": 108156946, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930745, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732332649, "oldest_key_time": 0, "file_creation_time": 1732332649, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 257829, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:30:53.216522 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257830.sst) /dev/shm/ToplingTemp/Topling-37170-EXn8LQ 2024/11/23-11:30:54.190796 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257830.sst): first pass time = 0.97's, 137.769'MB/sec 2024/11/23-11:30:55.652085 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257830.sst): old prealloc_size = 147639500, real_size = 31807104 2024/11/23-11:30:55.652624 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257830.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 938660 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1079 GB zip-val = 0.0298 GB avg-val = 114.99 avg-zval = 31.72 2024/11/23-11:30:55.654126 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257830.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:30:56.515537 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257830.sst): second pass time = 1.46's, 73.904'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 41.761'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3489.104'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.319'MB/sec write SST data time = 0.13's, 285.866'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 35945.780'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 73.904'MB/sec zip pipeline throughput = 73.904'MB/sec entries = 938660 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.99 avg-zval = 31.72 usrkeys = 938660 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.99 avg-zval = 31.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 938660 bytes = 3402656 } seq expand size = 6570620 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0188 GB value = 0.1079 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0298 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.4444 value = 3.6251 dict = 1.00 all = 3.7223 } Zip/UnZip{ index = 0.0957 value = 0.2759 dict = 1.00 all = 0.2687 } ---------------------------- total value len = 10.213101 GB avg = 0.161 KB (by entry num) total key len = 1.753790 GB avg = 0.028 KB total ukey len = 1.221427 GB avg = 0.016 KB total ukey num = 0.076727880 Billion total entry num = 0.063535609 Billion write speed all = 1.280661918 MB/sec (with seq num) write speed all = 1.226266744 MB/sec (without seq num) 2024/11/23-11:30:56.565287 139957525694016 [compaction_job.cc:1948] [default] [JOB 25225] Generated table #257830: 938660 keys, 36057656 bytes, temperature: kWarm 2024/11/23-11:30:56.565358 139957525694016 EVENT_LOG_v1 {"time_micros": 1732332656565320, "cf_name": "default", "job": 25225, "event": "table_file_creation", "file_number": 257830, "file_size": 36057656, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3621075956, "largest_seqno": 3952700283, "table_properties": {"data_size": 29774832, "index_size": 1797440, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26282480, "raw_average_key_size": 28, "raw_value_size": 107935348, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 938660, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732332653, "oldest_key_time": 0, "file_creation_time": 1732332653, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 257830, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:30:56.607154 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257831.sst) /dev/shm/ToplingTemp/Topling-37170-d7EuhN 2024/11/23-11:30:56.729895 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257831.sst): first pass time = 0.12's, 237.807'MB/sec 2024/11/23-11:30:57.009344 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257831.sst): old prealloc_size = 147639500, real_size = 8470720 2024/11/23-11:30:57.009524 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257831.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 202055 prefix = 4 raw-key = 0.0032 GB zip-key = 0.0004 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0235 GB zip-val = 0.0080 GB avg-val = 116.41 avg-zval = 39.74 2024/11/23-11:30:57.009739 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257831.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.029 GB 2024/11/23-11:30:57.187096 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00/257831.sst): second pass time = 0.28's, 84.364'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 53.029'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2408.415'MB/sec (index lex order gen) rebuild zvType time = 0.15's, 0.341'MB/sec write SST data time = 0.03's, 321.234'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.245'MB, throughput = 7590.409'MB/sec zip my value time = 0.28's, unzip length = 0.024'GB zip my value throughput = 84.364'MB/sec zip pipeline throughput = 84.364'MB/sec entries = 202055 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.41 avg-zval = 39.74 usrkeys = 202055 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.41 avg-zval = 39.74 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 202055 bytes = 757728 } seq expand size = 1414385 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0040 GB value = 0.0235 GB dict = 0.24 MB all = 0.0292 GB } __ZipSize{ index = 0.0004 GB value = 0.0080 GB dict = 0.24 MB all = 0.0094 GB } UnZip/Zip{ index = 10.3461 value = 2.9293 dict = 1.00 all = 3.0959 } Zip/UnZip{ index = 0.0967 value = 0.3414 dict = 1.00 all = 0.3230 } ---------------------------- total value len = 10.238036 GB avg = 0.161 KB (by entry num) total key len = 1.759447 GB avg = 0.028 KB total ukey len = 1.225468 GB avg = 0.016 KB total ukey num = 0.076929935 Billion total entry num = 0.063737664 Billion write speed all = 1.283843551 MB/sec (with seq num) write speed all = 1.229279312 MB/sec (without seq num) 2024/11/23-11:30:57.204534 139957525694016 [compaction_job.cc:1948] [default] [JOB 25225] Generated table #257831: 202055 keys, 9424803 bytes, temperature: kWarm 2024/11/23-11:30:57.204601 139957525694016 EVENT_LOG_v1 {"time_micros": 1732332657204566, "cf_name": "default", "job": 25225, "event": "table_file_creation", "file_number": 257831, "file_size": 9424803, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3952699332, "table_properties": {"data_size": 8029424, "index_size": 390592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 5657540, "raw_average_key_size": 28, "raw_value_size": 23520398, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 202055, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732332656, "oldest_key_time": 0, "file_creation_time": 1732332656, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 257831, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:30:57.216500 139957525694016 [compaction_job.cc:703] job-25225: subcompact[0], size: 0.455385 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25225/att-00 [257819,257820,257821,257822,257823,257824,257825,257826,257827,257828,257829,257830,257831] 2024/11/23-11:30:57.294870 139957525694016 (Original Log Time 2024/11/23-11:30:16.092451) EVENT_LOG_v1 {"time_micros": 1732332616092425, "job": 25225, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [257767], "files_L3": [257689, 257690, 257691, 257692, 257693, 257694, 257695, 257696, 257697, 257719, 257720, 257805], "score": 0.999537, "input_data_size": 475653909, "oldest_snapshot_seqno": 3953002514} 2024/11/23-11:30:57.294873 139957525694016 (Original Log Time 2024/11/23-11:30:57.293761) [compaction_job.cc:2022] [default] [JOB 25225] Compacted 1@2 + 12@3 files to L3 => 455385330 bytes 2024/11/23-11:30:57.294875 139957525694016 (Original Log Time 2024/11/23-11:30:57.294802) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 13 0 0 0] max score 0.02, MB/sec: 11.6 rd, 11.1 wr, level 3, files in(1, 12) out(13 +0 blob) MB in(27.1, 426.5 +0.0 blob) out(434.3 +0.0 blob), read-write-amplify(32.7) write-amplify(16.0) OK, records in: 11410186, records dropped: 30686 output_compression: Snappy 2024/11/23-11:30:57.294880 139957525694016 (Original Log Time 2024/11/23-11:30:57.294848) EVENT_LOG_v1 {"time_micros": 1732332657294827, "cf": "default", "job": 25225, "event": "compaction_finished", "compaction_time_micros": 41124060, "compaction_time_cpu_micros": 16898686, "output_level": 3, "num_output_files": 13, "total_output_size": 455385330, "num_input_records": 11410186, "num_output_records": 11379500, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 305053358, "file_range_sync_nanos": 0, "file_fsync_nanos": 8847, "file_prepare_write_nanos": 36005, "lsm_state": [0, 0, 0, 13, 0, 0, 0]} 2024/11/23-11:30:57.430044 139957525694016 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732332615,"executesMs":41334,"compactionJobId":25225,"attempt":0,"compactionInputRawBytes":1644149088,"compactionInputZipBytes":475653909,"compactionOutputRawBytes":0,"compactionOutputZipBytes":455385330,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-a3f9fe6c31be638dfe72ba993fd5ad79-0ffb6cd78f86512d-00"}}, response = 2024/11/23-11:30:57.430137 139957525694016 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-25225/att-00: olev 3, work 41.334 s, result 7.664 ms, install 1.136 ms, input{raw 1.531 GiB zip 453.619 MiB}