2024/11/22-23:04:51.034446 140024869324352 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00: fork to child time = 0.009626 sec 2024/11/22-23:04:51.035437 140024869324352 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00 2024/11/22-23:04:51.042968 140024869324352 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00 2024/11/22-23:04:51.044149 140024869324352 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-23:04:51.044394 140024869324352 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-23:04:51.091248 140024869324352 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-13508/att-00/MANIFEST-206347 2024/11/22-23:04:51.107944 140024869324352 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-13508/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 232860, last_sequence is 3630124908, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 232709 2024/11/22-23:04:51.107952 140024869324352 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-23:04:51.108044 140024869324352 [version_set.cc:5721] Creating manifest 232860 2024/11/22-23:04:51.141688 140024869324352 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-23:04:51.161358 140024869324352 [compaction_job.cc:2437] [default] [JOB 13508] Compacting 1@2 + 15@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/22-23:04:51.161377 140024869324352 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 10146 Base level 2, inputs: [232851(27MB)], [232774(34MB) 232775(35MB) 232776(35MB) 232777(35MB) 232778(34MB) 232779(33MB) 232780(35MB) 232781(34MB) 232782(36MB) 232783(36MB) 232784(35MB) 232785(36MB) 232786(35MB) 232787(36MB) 232814(37MB)] 2024/11/22-23:04:51.168602 140024869324352 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-23:04:51.168634 140024869324352 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-23:04:51.168654 140024869324352 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-23:04:51.168660 140024869324352 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-23:04:51.168670 140024869324352 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-23:04:51.685622 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232861.sst) /dev/shm/ToplingTemp/Topling-326070-Sjew64 2024/11/22-23:04:52.344941 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232861.sst): first pass time = 0.66's, 203.584'MB/sec 2024/11/22-23:04:54.464475 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232861.sst): old prealloc_size = 147639500, real_size = 32600144 2024/11/22-23:04:54.465029 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232861.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925009 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0306 GB avg-val = 117.10 avg-zval = 33.09 2024/11/22-23:04:54.465862 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232861.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:04:55.672032 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232861.sst): second pass time = 2.12's, 51.123'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.21's, 30.291'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3322.900'MB/sec (index lex order gen) rebuild zvType time = 1.07's, 0.216'MB/sec write SST data time = 0.14's, 266.500'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32354.543'MB/sec zip my value time = 2.12's, unzip length = 0.108'GB zip my value throughput = 51.123'MB/sec zip pipeline throughput = 96.470'MB/sec entries = 925009 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.10 avg-zval = 33.09 usrkeys = 925009 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.10 avg-zval = 33.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925009 bytes = 3121920 } seq expand size = 6475063 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5141 value = 3.5387 dict = 1.00 all = 3.6694 } Zip/UnZip{ index = 0.0951 value = 0.2826 dict = 1.00 all = 0.2725 } ---------------------------- total value len = 0.114793 GB avg = 0.124 KB (by entry num) total key len = 0.025900 GB avg = 0.028 KB total ukey len = 0.018500 GB avg = 0.020 KB total ukey num = 0.000925009 Billion total entry num = 0.000925009 Billion write speed all = 0.002750454 MB/sec (with seq num) write speed all = 0.002605787 MB/sec (without seq num) 2024/11/22-23:04:55.734740 140024869324352 [compaction_job.cc:1948] [default] [JOB 13508] Generated table #232861: 925009 keys, 36577686 bytes, temperature: kWarm 2024/11/22-23:04:55.734851 140024869324352 EVENT_LOG_v1 {"time_micros": 1732287895734776, "cf_name": "default", "job": 13508, "event": "table_file_creation", "file_number": 232861, "file_size": 36577686, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3513012344, "largest_seqno": 3630099196, "table_properties": {"data_size": 30609152, "index_size": 1759552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25900252, "raw_average_key_size": 28, "raw_value_size": 108317490, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925009, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287891, "oldest_key_time": 0, "file_creation_time": 1732287891, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232861, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:04:55.809451 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232862.sst) /dev/shm/ToplingTemp/Topling-326070-djxon6 2024/11/22-23:04:57.111914 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232862.sst): first pass time = 1.30's, 103.053'MB/sec 2024/11/22-23:04:59.984467 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232862.sst): old prealloc_size = 147639500, real_size = 33409280 2024/11/22-23:04:59.985057 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232862.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924777 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 117.14 avg-zval = 33.97 2024/11/22-23:04:59.985839 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232862.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:05:01.309281 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232862.sst): second pass time = 2.87's, 37.727'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.32's, 28.308'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3076.078'MB/sec (index lex order gen) rebuild zvType time = 1.09's, 0.212'MB/sec write SST data time = 0.23's, 161.017'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32656.540'MB/sec zip my value time = 2.87's, unzip length = 0.108'GB zip my value throughput = 37.727'MB/sec zip pipeline throughput = 136.981'MB/sec entries = 924777 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.14 avg-zval = 33.97 usrkeys = 924777 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.14 avg-zval = 33.97 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924777 bytes = 3236736 } seq expand size = 6473439 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5127 value = 3.4478 dict = 1.00 all = 3.5789 } Zip/UnZip{ index = 0.0951 value = 0.2900 dict = 1.00 all = 0.2794 } ---------------------------- total value len = 0.229590 GB avg = 0.124 KB (by entry num) total key len = 0.051794 GB avg = 0.028 KB total ukey len = 0.036996 GB avg = 0.020 KB total ukey num = 0.001849786 Billion total entry num = 0.001849786 Billion write speed all = 0.005500270 MB/sec (with seq num) write speed all = 0.005211005 MB/sec (without seq num) 2024/11/22-23:05:01.347722 140024869324352 [compaction_job.cc:1948] [default] [JOB 13508] Generated table #232862: 924777 keys, 37502044 bytes, temperature: kWarm 2024/11/22-23:05:01.347811 140024869324352 EVENT_LOG_v1 {"time_micros": 1732287901347760, "cf_name": "default", "job": 13508, "event": "table_file_creation", "file_number": 232862, "file_size": 37502044, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038037, "largest_seqno": 3630098257, "table_properties": {"data_size": 31418544, "index_size": 1759360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25893756, "raw_average_key_size": 28, "raw_value_size": 108324010, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924777, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287895, "oldest_key_time": 0, "file_creation_time": 1732287895, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232862, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:05:01.397747 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232863.sst) /dev/shm/ToplingTemp/Topling-326070-zEszA7 2024/11/22-23:05:02.851453 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232863.sst): first pass time = 1.45's, 92.332'MB/sec 2024/11/22-23:05:04.649471 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232863.sst): old prealloc_size = 147639500, real_size = 33153008 2024/11/22-23:05:04.650183 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232863.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925081 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 117.09 avg-zval = 33.69 2024/11/22-23:05:04.651001 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232863.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:05:06.399459 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232863.sst): second pass time = 1.80's, 60.267'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.75's, 21.218'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2559.780'MB/sec (index lex order gen) rebuild zvType time = 1.42's, 0.163'MB/sec write SST data time = 0.33's, 113.414'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 30164.304'MB/sec zip my value time = 1.80's, unzip length = 0.108'GB zip my value throughput = 60.267'MB/sec zip pipeline throughput = 67.849'MB/sec entries = 925081 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.09 avg-zval = 33.69 usrkeys = 925081 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.09 avg-zval = 33.69 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925081 bytes = 3122160 } seq expand size = 6475567 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5127 value = 3.4759 dict = 1.00 all = 3.6147 } Zip/UnZip{ index = 0.0951 value = 0.2877 dict = 1.00 all = 0.2766 } ---------------------------- total value len = 0.344381 GB avg = 0.124 KB (by entry num) total key len = 0.077696 GB avg = 0.028 KB total ukey len = 0.055497 GB avg = 0.020 KB total ukey num = 0.002774867 Billion total entry num = 0.002774867 Billion write speed all = 0.008249613 MB/sec (with seq num) write speed all = 0.007815729 MB/sec (without seq num) 2024/11/22-23:05:06.437382 140024869324352 [compaction_job.cc:1948] [default] [JOB 13508] Generated table #232863: 925081 keys, 37130712 bytes, temperature: kWarm 2024/11/22-23:05:06.437462 140024869324352 EVENT_LOG_v1 {"time_micros": 1732287906437421, "cf_name": "default", "job": 13508, "event": "table_file_creation", "file_number": 232863, "file_size": 37130712, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3497681670, "largest_seqno": 3630099500, "table_properties": {"data_size": 31161632, "index_size": 1759936, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25902268, "raw_average_key_size": 28, "raw_value_size": 108315590, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925081, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287901, "oldest_key_time": 0, "file_creation_time": 1732287901, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232863, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:05:06.509844 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232864.sst) /dev/shm/ToplingTemp/Topling-326070-zZjTp6 2024/11/22-23:05:07.746695 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232864.sst): first pass time = 1.24's, 108.519'MB/sec 2024/11/22-23:05:09.344870 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232864.sst): old prealloc_size = 147639500, real_size = 33518960 2024/11/22-23:05:09.345564 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232864.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924886 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 117.12 avg-zval = 34.09 2024/11/22-23:05:09.346373 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232864.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:05:10.467527 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232864.sst): second pass time = 1.60's, 67.817'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.12's, 33.503'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2615.043'MB/sec (index lex order gen) rebuild zvType time = 0.97's, 0.239'MB/sec write SST data time = 0.15's, 245.207'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35953.247'MB/sec zip my value time = 1.60's, unzip length = 0.108'GB zip my value throughput = 67.817'MB/sec zip pipeline throughput = 67.817'MB/sec entries = 924886 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.12 avg-zval = 34.09 usrkeys = 924886 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.12 avg-zval = 34.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924886 bytes = 3237120 } seq expand size = 6474202 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5173 value = 3.4356 dict = 1.00 all = 3.5685 } Zip/UnZip{ index = 0.0951 value = 0.2911 dict = 1.00 all = 0.2802 } ---------------------------- total value len = 0.459176 GB avg = 0.124 KB (by entry num) total key len = 0.103593 GB avg = 0.028 KB total ukey len = 0.073995 GB avg = 0.020 KB total ukey num = 0.003699753 Billion total entry num = 0.003699753 Billion write speed all = 0.010998600 MB/sec (with seq num) write speed all = 0.010420145 MB/sec (without seq num) 2024/11/22-23:05:10.501302 140024869324352 [compaction_job.cc:1948] [default] [JOB 13508] Generated table #232864: 924886 keys, 37611990 bytes, temperature: kWarm 2024/11/22-23:05:10.501409 140024869324352 EVENT_LOG_v1 {"time_micros": 1732287910501331, "cf_name": "default", "job": 13508, "event": "table_file_creation", "file_number": 232864, "file_size": 37611990, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038075, "largest_seqno": 3630098292, "table_properties": {"data_size": 31528800, "index_size": 1758784, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25896808, "raw_average_key_size": 28, "raw_value_size": 108320992, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924886, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287906, "oldest_key_time": 0, "file_creation_time": 1732287906, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232864, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:05:10.538570 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232865.sst) /dev/shm/ToplingTemp/Topling-326070-Ae1yf7 2024/11/22-23:05:11.593600 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232865.sst): first pass time = 1.05's, 127.221'MB/sec 2024/11/22-23:05:13.898703 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232865.sst): old prealloc_size = 147639500, real_size = 32799760 2024/11/22-23:05:13.899266 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232865.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929163 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0308 GB avg-val = 116.45 avg-zval = 33.15 2024/11/22-23:05:13.900026 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232865.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:05:15.011436 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232865.sst): second pass time = 2.30's, 46.966'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.11's, 33.062'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3254.966'MB/sec (index lex order gen) rebuild zvType time = 0.98's, 0.236'MB/sec write SST data time = 0.13's, 287.307'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 28303.189'MB/sec zip my value time = 2.30's, unzip length = 0.108'GB zip my value throughput = 46.966'MB/sec zip pipeline throughput = 143.779'MB/sec entries = 929163 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.45 avg-zval = 33.15 usrkeys = 929163 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.45 avg-zval = 33.15 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929163 bytes = 3135936 } seq expand size = 6504141 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5109 value = 3.5131 dict = 1.00 all = 3.6484 } Zip/UnZip{ index = 0.0951 value = 0.2846 dict = 1.00 all = 0.2741 } ---------------------------- total value len = 0.573882 GB avg = 0.124 KB (by entry num) total key len = 0.129610 GB avg = 0.028 KB total ukey len = 0.092578 GB avg = 0.020 KB total ukey num = 0.004628916 Billion total entry num = 0.004628916 Billion write speed all = 0.013747607 MB/sec (with seq num) write speed all = 0.013023942 MB/sec (without seq num) 2024/11/22-23:05:15.048120 140024869324352 [compaction_job.cc:1948] [default] [JOB 13508] Generated table #232865: 929163 keys, 36788408 bytes, temperature: kWarm 2024/11/22-23:05:15.048180 140024869324352 EVENT_LOG_v1 {"time_micros": 1732287915048149, "cf_name": "default", "job": 13508, "event": "table_file_creation", "file_number": 232865, "file_size": 36788408, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3523718385, "largest_seqno": 3630099560, "table_properties": {"data_size": 30799296, "index_size": 1768000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26016564, "raw_average_key_size": 28, "raw_value_size": 108201246, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929163, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287910, "oldest_key_time": 0, "file_creation_time": 1732287910, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232865, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:05:15.083537 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232866.sst) /dev/shm/ToplingTemp/Topling-326070-noysC4 2024/11/22-23:05:15.952411 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232866.sst): first pass time = 0.87's, 154.479'MB/sec 2024/11/22-23:05:17.639270 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232866.sst): old prealloc_size = 147639500, real_size = 31395888 2024/11/22-23:05:17.639836 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232866.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924846 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0294 GB avg-val = 117.12 avg-zval = 31.79 2024/11/22-23:05:17.640605 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232866.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:05:18.500757 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232866.sst): second pass time = 1.69's, 64.239'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 41.195'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3220.513'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.315'MB/sec write SST data time = 0.13's, 281.568'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33751.647'MB/sec zip my value time = 1.69's, unzip length = 0.108'GB zip my value throughput = 64.239'MB/sec zip pipeline throughput = 69.441'MB/sec entries = 924846 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.12 avg-zval = 31.79 usrkeys = 924846 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.12 avg-zval = 31.79 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924846 bytes = 3236976 } seq expand size = 6473922 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.5131 value = 3.6838 dict = 1.00 all = 3.7820 } Zip/UnZip{ index = 0.0951 value = 0.2715 dict = 1.00 all = 0.2644 } ---------------------------- total value len = 0.688678 GB avg = 0.124 KB (by entry num) total key len = 0.155505 GB avg = 0.028 KB total ukey len = 0.111075 GB avg = 0.020 KB total ukey num = 0.005553762 Billion total entry num = 0.005553762 Billion write speed all = 0.016495877 MB/sec (with seq num) write speed all = 0.015627684 MB/sec (without seq num) 2024/11/22-23:05:18.532421 140024869324352 [compaction_job.cc:1948] [default] [JOB 13508] Generated table #232866: 924846 keys, 35488738 bytes, temperature: kWarm 2024/11/22-23:05:18.532478 140024869324352 EVENT_LOG_v1 {"time_micros": 1732287918532448, "cf_name": "default", "job": 13508, "event": "table_file_creation", "file_number": 232866, "file_size": 35488738, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038106, "largest_seqno": 3630098339, "table_properties": {"data_size": 29405088, "index_size": 1759424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25895688, "raw_average_key_size": 28, "raw_value_size": 108322160, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924846, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287915, "oldest_key_time": 0, "file_creation_time": 1732287915, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232866, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:05:18.570103 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232867.sst) /dev/shm/ToplingTemp/Topling-326070-PwjPj4 2024/11/22-23:05:19.453666 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232867.sst): first pass time = 0.88's, 151.911'MB/sec 2024/11/22-23:05:20.963279 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232867.sst): old prealloc_size = 147639500, real_size = 32743040 2024/11/22-23:05:20.963834 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232867.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926268 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0307 GB avg-val = 116.90 avg-zval = 33.18 2024/11/22-23:05:20.964593 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232867.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:05:21.849036 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232867.sst): second pass time = 1.51's, 71.761'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 41.592'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3367.372'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.305'MB/sec write SST data time = 0.13's, 291.243'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 26886.174'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.761'MB/sec zip pipeline throughput = 115.511'MB/sec entries = 926268 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.90 avg-zval = 33.18 usrkeys = 926268 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.90 avg-zval = 33.18 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926268 bytes = 3241952 } seq expand size = 6483876 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.4460 value = 3.5228 dict = 1.00 all = 3.6433 } Zip/UnZip{ index = 0.0957 value = 0.2839 dict = 1.00 all = 0.2745 } ---------------------------- total value len = 0.803444 GB avg = 0.124 KB (by entry num) total key len = 0.181441 GB avg = 0.028 KB total ukey len = 0.129601 GB avg = 0.020 KB total ukey num = 0.006480030 Billion total entry num = 0.006480030 Billion write speed all = 0.019244017 MB/sec (with seq num) write speed all = 0.018231092 MB/sec (without seq num) 2024/11/22-23:05:21.883841 140024869324352 [compaction_job.cc:1948] [default] [JOB 13508] Generated table #232867: 926268 keys, 36839642 bytes, temperature: kWarm 2024/11/22-23:05:21.883909 140024869324352 EVENT_LOG_v1 {"time_micros": 1732287921883866, "cf_name": "default", "job": 13508, "event": "table_file_creation", "file_number": 232867, "file_size": 36839642, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038136, "largest_seqno": 3630098629, "table_properties": {"data_size": 30737840, "index_size": 1773440, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25935504, "raw_average_key_size": 28, "raw_value_size": 108282236, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926268, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287918, "oldest_key_time": 0, "file_creation_time": 1732287918, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232867, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:05:21.931972 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232868.sst) /dev/shm/ToplingTemp/Topling-326070-AWmSX2 2024/11/22-23:05:22.846987 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232868.sst): first pass time = 0.91's, 146.689'MB/sec 2024/11/22-23:05:24.402581 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232868.sst): old prealloc_size = 147639500, real_size = 32212112 2024/11/22-23:05:24.403135 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232868.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924818 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0302 GB avg-val = 117.13 avg-zval = 32.67 2024/11/22-23:05:24.403871 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232868.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:05:25.290277 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232868.sst): second pass time = 1.55's, 69.667'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 40.768'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3305.067'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.307'MB/sec write SST data time = 0.13's, 271.933'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 24464.852'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 69.667'MB/sec zip pipeline throughput = 139.350'MB/sec entries = 924818 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.13 avg-zval = 32.67 usrkeys = 924818 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.13 avg-zval = 32.67 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924818 bytes = 3121280 } seq expand size = 6473726 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.4444 value = 3.5857 dict = 1.00 all = 3.7088 } Zip/UnZip{ index = 0.0957 value = 0.2789 dict = 1.00 all = 0.2696 } ---------------------------- total value len = 0.918241 GB avg = 0.124 KB (by entry num) total key len = 0.207336 GB avg = 0.028 KB total ukey len = 0.148097 GB avg = 0.020 KB total ukey num = 0.007404848 Billion total entry num = 0.007404848 Billion write speed all = 0.021991561 MB/sec (with seq num) write speed all = 0.020834151 MB/sec (without seq num) 2024/11/22-23:05:25.324258 140024869324352 [compaction_job.cc:1948] [default] [JOB 13508] Generated table #232868: 924818 keys, 36189370 bytes, temperature: kWarm 2024/11/22-23:05:25.324316 140024869324352 EVENT_LOG_v1 {"time_micros": 1732287925324286, "cf_name": "default", "job": 13508, "event": "table_file_creation", "file_number": 232868, "file_size": 36189370, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3499102480, "largest_seqno": 3630099654, "table_properties": {"data_size": 30209792, "index_size": 1770944, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25894904, "raw_average_key_size": 28, "raw_value_size": 108322956, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924818, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287921, "oldest_key_time": 0, "file_creation_time": 1732287921, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232868, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:05:25.361306 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232869.sst) /dev/shm/ToplingTemp/Topling-326070-9eTsP5 2024/11/22-23:05:26.282014 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232869.sst): first pass time = 0.92's, 145.783'MB/sec 2024/11/22-23:05:27.856505 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232869.sst): old prealloc_size = 147639500, real_size = 33146272 2024/11/22-23:05:27.857102 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232869.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925152 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 117.08 avg-zval = 33.66 2024/11/22-23:05:27.859136 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232869.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:05:28.786137 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232869.sst): second pass time = 1.57's, 68.822'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.188'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3066.408'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.289'MB/sec write SST data time = 0.13's, 289.408'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32085.493'MB/sec zip my value time = 1.57's, unzip length = 0.108'GB zip my value throughput = 68.822'MB/sec zip pipeline throughput = 104.778'MB/sec entries = 925152 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.08 avg-zval = 33.66 usrkeys = 925152 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.08 avg-zval = 33.66 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925152 bytes = 3353696 } seq expand size = 6476064 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4440 value = 3.4779 dict = 1.00 all = 3.5926 } Zip/UnZip{ index = 0.0957 value = 0.2875 dict = 1.00 all = 0.2783 } ---------------------------- total value len = 1.033030 GB avg = 0.124 KB (by entry num) total key len = 0.233240 GB avg = 0.028 KB total ukey len = 0.166600 GB avg = 0.020 KB total ukey num = 0.008330000 Billion total entry num = 0.008330000 Billion write speed all = 0.024738756 MB/sec (with seq num) write speed all = 0.023436829 MB/sec (without seq num) 2024/11/22-23:05:28.818772 140024869324352 [compaction_job.cc:1948] [default] [JOB 13508] Generated table #232869: 925152 keys, 37359216 bytes, temperature: kWarm 2024/11/22-23:05:28.818839 140024869324352 EVENT_LOG_v1 {"time_micros": 1732287928818808, "cf_name": "default", "job": 13508, "event": "table_file_creation", "file_number": 232869, "file_size": 37359216, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868609, "largest_seqno": 3630099908, "table_properties": {"data_size": 31143184, "index_size": 1771648, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25904256, "raw_average_key_size": 28, "raw_value_size": 108313596, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925152, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287925, "oldest_key_time": 0, "file_creation_time": 1732287925, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232869, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:05:28.854764 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232870.sst) /dev/shm/ToplingTemp/Topling-326070-r33jm7 2024/11/22-23:05:29.640416 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232870.sst): first pass time = 0.79's, 170.845'MB/sec 2024/11/22-23:05:31.273971 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232870.sst): old prealloc_size = 147639500, real_size = 37066512 2024/11/22-23:05:31.274501 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232870.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926508 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0351 GB avg-val = 116.86 avg-zval = 37.85 2024/11/22-23:05:31.274791 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232870.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:05:31.953806 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232870.sst): second pass time = 1.63's, 66.308'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.68's, 57.093'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3463.617'MB/sec (index lex order gen) rebuild zvType time = 0.55's, 0.418'MB/sec write SST data time = 0.13's, 310.432'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33745.222'MB/sec zip my value time = 1.63's, unzip length = 0.108'GB zip my value throughput = 66.308'MB/sec zip pipeline throughput = 68.819'MB/sec entries = 926508 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.86 avg-zval = 37.85 usrkeys = 926508 avg-key = 20.00 avg-zkey = 1.90 avg-val = 118.59 avg-zval = 37.85 TagRS{ kinds = RS_Key0_TagN bytes = 144880 } TagArray{ size = 228498 bytes = 742640 } seq expand size = 1599486 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0351 GB dict = 1.08 MB all = 0.0388 GB } UnZip/Zip{ index = 10.5136 value = 3.0872 dict = 1.00 all = 3.4581 } Zip/UnZip{ index = 0.0951 value = 0.3239 dict = 1.00 all = 0.2892 } ---------------------------- total value len = 1.142905 GB avg = 0.123 KB (by entry num) total key len = 0.259182 GB avg = 0.028 KB total ukey len = 0.185130 GB avg = 0.020 KB total ukey num = 0.009256508 Billion total entry num = 0.009256508 Billion write speed all = 0.027390485 MB/sec (with seq num) write speed all = 0.025943841 MB/sec (without seq num) 2024/11/22-23:05:31.991915 140024869324352 [compaction_job.cc:1948] [default] [JOB 13508] Generated table #232870: 926508 keys, 38812441 bytes, temperature: kWarm 2024/11/22-23:05:31.991978 140024869324352 EVENT_LOG_v1 {"time_micros": 1732287931991947, "cf_name": "default", "job": 13508, "event": "table_file_creation", "file_number": 232870, "file_size": 38812441, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700833, "table_properties": {"data_size": 35072192, "index_size": 1762496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25942224, "raw_average_key_size": 28, "raw_value_size": 108275640, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926508, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287928, "oldest_key_time": 0, "file_creation_time": 1732287928, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232870, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:05:32.035024 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232871.sst) /dev/shm/ToplingTemp/Topling-326070-L1U4Q6 2024/11/22-23:05:32.858331 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232871.sst): first pass time = 0.82's, 163.032'MB/sec 2024/11/22-23:05:34.539767 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232871.sst): old prealloc_size = 147639500, real_size = 36503392 2024/11/22-23:05:34.540330 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232871.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927082 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.77 avg-zval = 37.21 2024/11/22-23:05:34.540372 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232871.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:05:35.182439 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232871.sst): second pass time = 1.68's, 64.408'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 58.136'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3339.886'MB/sec (index lex order gen) rebuild zvType time = 0.52's, 0.448'MB/sec write SST data time = 0.13's, 298.268'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31066.835'MB/sec zip my value time = 1.68's, unzip length = 0.108'GB zip my value throughput = 64.408'MB/sec zip pipeline throughput = 69.840'MB/sec entries = 927082 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.77 avg-zval = 37.21 usrkeys = 927082 avg-key = 20.00 avg-zkey = 1.91 avg-val = 116.77 avg-zval = 37.21 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 10 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4439 value = 3.1383 dict = 1.00 all = 3.5924 } Zip/UnZip{ index = 0.0957 value = 0.3186 dict = 1.00 all = 0.2784 } ---------------------------- total value len = 1.251165 GB avg = 0.123 KB (by entry num) total key len = 0.285141 GB avg = 0.028 KB total ukey len = 0.203672 GB avg = 0.020 KB total ukey num = 0.010183590 Billion total entry num = 0.010183590 Billion write speed all = 0.030010604 MB/sec (with seq num) write speed all = 0.028419172 MB/sec (without seq num) 2024/11/22-23:05:35.217650 140024869324352 [compaction_job.cc:1948] [default] [JOB 13508] Generated table #232871: 927082 keys, 37361346 bytes, temperature: kWarm 2024/11/22-23:05:35.217713 140024869324352 EVENT_LOG_v1 {"time_micros": 1732287935217682, "cf_name": "default", "job": 13508, "event": "table_file_creation", "file_number": 232871, "file_size": 37361346, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34496080, "index_size": 1775360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25958296, "raw_average_key_size": 28, "raw_value_size": 108259524, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927082, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287932, "oldest_key_time": 0, "file_creation_time": 1732287932, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232871, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:05:35.255662 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232872.sst) /dev/shm/ToplingTemp/Topling-326070-ftcdZ2 2024/11/22-23:05:36.101248 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232872.sst): first pass time = 0.85's, 158.735'MB/sec 2024/11/22-23:05:37.783771 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232872.sst): old prealloc_size = 147639500, real_size = 36187680 2024/11/22-23:05:37.784335 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232872.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923893 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0342 GB avg-val = 117.27 avg-zval = 37.02 2024/11/22-23:05:37.784700 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232872.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:05:38.538025 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232872.sst): second pass time = 1.68's, 64.421'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.75's, 50.973'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3273.220'MB/sec (index lex order gen) rebuild zvType time = 0.63's, 0.368'MB/sec write SST data time = 0.13's, 304.181'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 27132.386'MB/sec zip my value time = 1.68's, unzip length = 0.108'GB zip my value throughput = 64.421'MB/sec zip pipeline throughput = 123.359'MB/sec entries = 923893 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.27 avg-zval = 37.02 usrkeys = 923893 avg-key = 20.00 avg-zkey = 1.90 avg-val = 120.09 avg-zval = 37.02 TagRS{ kinds = RS_Key0_TagN bytes = 144440 } TagArray{ size = 371214 bytes = 1252864 } seq expand size = 2598498 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0342 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.5118 value = 3.1682 dict = 1.00 all = 3.4911 } Zip/UnZip{ index = 0.0951 value = 0.3156 dict = 1.00 all = 0.2864 } ---------------------------- total value len = 1.362112 GB avg = 0.123 KB (by entry num) total key len = 0.311010 GB avg = 0.028 KB total ukey len = 0.222150 GB avg = 0.020 KB total ukey num = 0.011107483 Billion total entry num = 0.011107483 Billion write speed all = 0.032681067 MB/sec (with seq num) write speed all = 0.030945368 MB/sec (without seq num) 2024/11/22-23:05:38.576914 140024869324352 [compaction_job.cc:1948] [default] [JOB 13508] Generated table #232872: 923893 keys, 38445809 bytes, temperature: kWarm 2024/11/22-23:05:38.576988 140024869324352 EVENT_LOG_v1 {"time_micros": 1732287938576955, "cf_name": "default", "job": 13508, "event": "table_file_creation", "file_number": 232872, "file_size": 38445809, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3381656393, "table_properties": {"data_size": 34198672, "index_size": 1757824, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25869004, "raw_average_key_size": 28, "raw_value_size": 108348726, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923893, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287935, "oldest_key_time": 0, "file_creation_time": 1732287935, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232872, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:05:38.612978 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232873.sst) /dev/shm/ToplingTemp/Topling-326070-HHN954 2024/11/22-23:05:39.591734 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232873.sst): first pass time = 0.98's, 137.137'MB/sec 2024/11/22-23:05:40.933693 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232873.sst): old prealloc_size = 147639500, real_size = 33178432 2024/11/22-23:05:40.934255 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232873.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925358 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 117.04 avg-zval = 33.70 2024/11/22-23:05:40.935029 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232873.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:05:41.853158 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232873.sst): second pass time = 1.34's, 80.795'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.538'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3247.765'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.292'MB/sec write SST data time = 0.13's, 293.170'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35161.672'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.795'MB/sec zip pipeline throughput = 80.795'MB/sec entries = 925358 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.04 avg-zval = 33.70 usrkeys = 925358 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.04 avg-zval = 33.70 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925358 bytes = 3238768 } seq expand size = 6477506 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5128 value = 3.4729 dict = 1.00 all = 3.6010 } Zip/UnZip{ index = 0.0951 value = 0.2879 dict = 1.00 all = 0.2777 } ---------------------------- total value len = 1.476897 GB avg = 0.123 KB (by entry num) total key len = 0.336920 GB avg = 0.028 KB total ukey len = 0.240657 GB avg = 0.020 KB total ukey num = 0.012032841 Billion total entry num = 0.012032841 Billion write speed all = 0.035426974 MB/sec (with seq num) write speed all = 0.033546797 MB/sec (without seq num) 2024/11/22-23:05:41.887119 140024869324352 [compaction_job.cc:1948] [default] [JOB 13508] Generated table #232873: 925358 keys, 37272390 bytes, temperature: kWarm 2024/11/22-23:05:41.887185 140024869324352 EVENT_LOG_v1 {"time_micros": 1732287941887151, "cf_name": "default", "job": 13508, "event": "table_file_creation", "file_number": 232873, "file_size": 37272390, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3381656394, "largest_seqno": 3630099926, "table_properties": {"data_size": 31186480, "index_size": 1760448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25910024, "raw_average_key_size": 28, "raw_value_size": 108307836, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925358, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287938, "oldest_key_time": 0, "file_creation_time": 1732287938, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232873, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:05:41.923986 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232874.sst) /dev/shm/ToplingTemp/Topling-326070-W6Zqp4 2024/11/22-23:05:42.742982 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232874.sst): first pass time = 0.82's, 163.888'MB/sec 2024/11/22-23:05:44.083440 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232874.sst): old prealloc_size = 147639500, real_size = 33184320 2024/11/22-23:05:44.084006 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232874.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931179 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0312 GB avg-val = 116.14 avg-zval = 33.48 2024/11/22-23:05:44.084764 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232874.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:05:45.023937 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232874.sst): second pass time = 1.34's, 80.715'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.779'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3251.519'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.287'MB/sec write SST data time = 0.13's, 292.064'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33709.686'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.715'MB/sec zip pipeline throughput = 80.715'MB/sec entries = 931179 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.14 avg-zval = 33.48 usrkeys = 931179 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.14 avg-zval = 33.48 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931179 bytes = 3375536 } seq expand size = 6518253 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5135 value = 3.4684 dict = 1.00 all = 3.5876 } Zip/UnZip{ index = 0.0951 value = 0.2883 dict = 1.00 all = 0.2787 } ---------------------------- total value len = 1.591561 GB avg = 0.123 KB (by entry num) total key len = 0.362993 GB avg = 0.028 KB total ukey len = 0.259280 GB avg = 0.020 KB total ukey num = 0.012964020 Billion total entry num = 0.012964020 Billion write speed all = 0.038173428 MB/sec (with seq num) write speed all = 0.036147876 MB/sec (without seq num) 2024/11/22-23:05:45.060425 140024869324352 [compaction_job.cc:1948] [default] [JOB 13508] Generated table #232874: 931179 keys, 37411354 bytes, temperature: kWarm 2024/11/22-23:05:45.060489 140024869324352 EVENT_LOG_v1 {"time_micros": 1732287945060458, "cf_name": "default", "job": 13508, "event": "table_file_creation", "file_number": 232874, "file_size": 37411354, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701386, "largest_seqno": 3630099972, "table_properties": {"data_size": 31179952, "index_size": 1771392, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26073012, "raw_average_key_size": 28, "raw_value_size": 108144782, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931179, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287941, "oldest_key_time": 0, "file_creation_time": 1732287941, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232874, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:05:45.112337 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232875.sst) /dev/shm/ToplingTemp/Topling-326070-w8W6G6 2024/11/22-23:05:45.980039 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232875.sst): first pass time = 0.87's, 154.690'MB/sec 2024/11/22-23:05:47.324178 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232875.sst): old prealloc_size = 147639500, real_size = 34827264 2024/11/22-23:05:47.324709 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232875.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930408 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0328 GB avg-val = 116.26 avg-zval = 35.27 2024/11/22-23:05:47.325516 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232875.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:05:48.216588 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232875.sst): second pass time = 1.34's, 80.538'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 43.761'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3498.111'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.304'MB/sec write SST data time = 0.13's, 305.977'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34253.715'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.538'MB/sec zip pipeline throughput = 80.538'MB/sec entries = 930408 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.26 avg-zval = 35.27 usrkeys = 930408 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.26 avg-zval = 35.27 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930408 bytes = 3372736 } seq expand size = 6512856 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0328 GB dict = 1.08 MB all = 0.0391 GB } UnZip/Zip{ index = 10.4452 value = 3.2965 dict = 1.00 all = 3.4369 } Zip/UnZip{ index = 0.0957 value = 0.3034 dict = 1.00 all = 0.2910 } ---------------------------- total value len = 1.706240 GB avg = 0.123 KB (by entry num) total key len = 0.389044 GB avg = 0.028 KB total ukey len = 0.277889 GB avg = 0.020 KB total ukey num = 0.013894428 Billion total entry num = 0.013894428 Billion write speed all = 0.040919418 MB/sec (with seq num) write speed all = 0.038748631 MB/sec (without seq num) 2024/11/22-23:05:48.251962 140024869324352 [compaction_job.cc:1948] [default] [JOB 13508] Generated table #232875: 930408 keys, 39052046 bytes, temperature: kWarm 2024/11/22-23:05:48.252025 140024869324352 EVENT_LOG_v1 {"time_micros": 1732287948251993, "cf_name": "default", "job": 13508, "event": "table_file_creation", "file_number": 232875, "file_size": 39052046, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3281472296, "largest_seqno": 3630098871, "table_properties": {"data_size": 32812976, "index_size": 1781504, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26051424, "raw_average_key_size": 28, "raw_value_size": 108166380, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930408, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287945, "oldest_key_time": 0, "file_creation_time": 1732287945, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232875, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:05:48.294659 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232876.sst) /dev/shm/ToplingTemp/Topling-326070-8S8ww3 2024/11/22-23:05:48.371693 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232876.sst): first pass time = 0.08's, 236.873'MB/sec 2024/11/22-23:05:48.532555 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232876.sst): old prealloc_size = 147639500, real_size = 5412032 2024/11/22-23:05:48.532672 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232876.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 124920 prefix = 4 raw-key = 0.0020 GB zip-key = 0.0002 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0147 GB zip-val = 0.0051 GB avg-val = 118.00 avg-zval = 41.15 2024/11/22-23:05:48.532816 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232876.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.018 GB 2024/11/22-23:05:48.653891 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00/232876.sst): second pass time = 0.16's, 91.949'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.12's, 48.854'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2558.747'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.301'MB/sec write SST data time = 0.02's, 339.288'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.155'MB, throughput = 5986.136'MB/sec zip my value time = 0.16's, unzip length = 0.015'GB zip my value throughput = 91.949'MB/sec zip pipeline throughput = 91.949'MB/sec entries = 124920 avg-key = 28.00 avg-zkey = 1.92 avg-val = 118.00 avg-zval = 41.15 usrkeys = 124920 avg-key = 20.00 avg-zkey = 1.92 avg-val = 125.00 avg-zval = 41.15 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 124920 bytes = 390384 } seq expand size = 874440 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0025 GB value = 0.0147 GB dict = 0.15 MB all = 0.0182 GB } __ZipSize{ index = 0.0002 GB value = 0.0051 GB dict = 0.15 MB all = 0.0059 GB } UnZip/Zip{ index = 10.3906 value = 2.8677 dict = 1.00 all = 3.0771 } Zip/UnZip{ index = 0.0962 value = 0.3487 dict = 1.00 all = 0.3250 } ---------------------------- total value len = 1.721855 GB avg = 0.123 KB (by entry num) total key len = 0.392542 GB avg = 0.028 KB total ukey len = 0.280387 GB avg = 0.020 KB total ukey num = 0.014019348 Billion total entry num = 0.014019348 Billion write speed all = 0.041292324 MB/sec (with seq num) write speed all = 0.039102039 MB/sec (without seq num) 2024/11/22-23:05:48.665162 140024869324352 [compaction_job.cc:1948] [default] [JOB 13508] Generated table #232876: 124920 keys, 5927155 bytes, temperature: kWarm 2024/11/22-23:05:48.665219 140024869324352 EVENT_LOG_v1 {"time_micros": 1732287948665189, "cf_name": "default", "job": 13508, "event": "table_file_creation", "file_number": 232876, "file_size": 5927155, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3574114653, "largest_seqno": 3599571143, "table_properties": {"data_size": 5140144, "index_size": 240448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3497760, "raw_average_key_size": 28, "raw_value_size": 14740560, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 124920, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287948, "oldest_key_time": 0, "file_creation_time": 1732287948, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232876, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:05:48.677043 140024869324352 [compaction_job.cc:703] job-13508: subcompact[0], size: 0.565770 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13508/att-00 [232861,232862,232863,232864,232865,232866,232867,232868,232869,232870,232871,232872,232873,232874,232875,232876] 2024/11/22-23:05:48.801276 140024869324352 (Original Log Time 2024/11/22-23:04:51.161429) EVENT_LOG_v1 {"time_micros": 1732287891161389, "job": 13508, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [232851], "files_L3": [232774, 232775, 232776, 232777, 232778, 232779, 232780, 232781, 232782, 232783, 232784, 232785, 232786, 232787, 232814], "score": 0.998627, "input_data_size": 589611238, "oldest_snapshot_seqno": 3630124748} 2024/11/22-23:05:48.801280 140024869324352 (Original Log Time 2024/11/22-23:05:48.800435) [compaction_job.cc:2022] [default] [JOB 13508] Compacted 1@2 + 15@3 files to L3 => 565770347 bytes 2024/11/22-23:05:48.801281 140024869324352 (Original Log Time 2024/11/22-23:05:48.801210) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 16 0 0 0] max score 0.02, MB/sec: 10.3 rd, 9.8 wr, level 3, files in(1, 15) out(16 +0 blob) MB in(27.1, 535.2 +0.0 blob) out(539.6 +0.0 blob), read-write-amplify(40.6) write-amplify(19.9) OK, records in: 14126974, records dropped: 107626 output_compression: Snappy 2024/11/22-23:05:48.801288 140024869324352 (Original Log Time 2024/11/22-23:05:48.801257) EVENT_LOG_v1 {"time_micros": 1732287948801235, "cf": "default", "job": 13508, "event": "compaction_finished", "compaction_time_micros": 57515621, "compaction_time_cpu_micros": 24182460, "output_level": 3, "num_output_files": 16, "total_output_size": 565770347, "num_input_records": 14126974, "num_output_records": 14019348, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 386296086, "file_range_sync_nanos": 0, "file_fsync_nanos": 15744, "file_prepare_write_nanos": 37095, "lsm_state": [0, 0, 0, 16, 0, 0, 0]} 2024/11/22-23:05:49.009595 140024869324352 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732287891,"executesMs":57760,"compactionJobId":13508,"attempt":0,"compactionInputRawBytes":2046788360,"compactionInputZipBytes":589611238,"compactionOutputRawBytes":0,"compactionOutputZipBytes":565770347,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-c3ee84394616bf329605607397e923c5-948d5e0801b8f7c1-00"}}, response = 2024/11/22-23:05:49.009665 140024869324352 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-13508/att-00: olev 3, work 57.761 s, result 4.942 ms, install 0.874 ms, input{raw 1.906 GiB zip 562.297 MiB}