2024/11/22-16:53:03.918102 140024735041088 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00: fork to child time = 0.012701 sec 2024/11/22-16:53:03.919313 140024735041088 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00 2024/11/22-16:53:03.929223 140024735041088 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00 2024/11/22-16:53:03.930808 140024735041088 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-16:53:03.931181 140024735041088 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-16:53:03.989878 140024735041088 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-05181/att-00/MANIFEST-206347 2024/11/22-16:53:04.025348 140024735041088 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-05181/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 215841, last_sequence is 3397818394, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 215696 2024/11/22-16:53:04.025360 140024735041088 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-16:53:04.025519 140024735041088 [version_set.cc:5721] Creating manifest 215841 2024/11/22-16:53:04.059434 140024735041088 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-16:53:04.082455 140024735041088 [compaction_job.cc:2437] [default] [JOB 5181] Compacting 1@2 + 14@3 files to L3, score 0.90, subcompactions 1 : 1 2024/11/22-16:53:04.082477 140024735041088 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 4019 Base level 2, inputs: [215825(27MB)], [215775(35MB) 215776(35MB) 215777(36MB) 215778(36MB) 215779(35MB) 215780(35MB) 215781(36MB) 215782(39MB) 215783(38MB) 215784(37MB) 215785(35MB) 215786(36MB) 215787(36MB) 215794(37MB)] 2024/11/22-16:53:04.095372 140024735041088 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-16:53:04.095409 140024735041088 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-16:53:04.095433 140024735041088 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-16:53:04.095441 140024735041088 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-16:53:04.095455 140024735041088 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-16:53:04.604535 140024735041088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215842.sst) /dev/shm/ToplingTemp/Topling-207718-IDAZ4S 2024/11/22-16:53:05.475155 140024735041088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215842.sst): first pass time = 0.87's, 154.173'MB/sec 2024/11/22-16:53:08.502128 140024735041088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215842.sst): old prealloc_size = 147639500, real_size = 33631952 2024/11/22-16:53:08.502687 140024735041088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215842.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921424 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0316 GB avg-val = 117.66 avg-zval = 34.33 2024/11/22-16:53:08.503476 140024735041088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215842.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:53:09.751013 140024735041088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215842.sst): second pass time = 3.03's, 35.833'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.25's, 30.109'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3278.615'MB/sec (index lex order gen) rebuild zvType time = 1.02's, 0.226'MB/sec write SST data time = 0.23's, 162.893'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 25357.619'MB/sec zip my value time = 3.03's, unzip length = 0.108'GB zip my value throughput = 35.833'MB/sec zip pipeline throughput = 137.414'MB/sec entries = 921424 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.66 avg-zval = 34.33 usrkeys = 921424 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.66 avg-zval = 34.33 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921424 bytes = 3109824 } seq expand size = 6449968 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.4430 value = 3.4270 dict = 1.00 all = 3.5695 } Zip/UnZip{ index = 0.0958 value = 0.2918 dict = 1.00 all = 0.2801 } ---------------------------- total value len = 0.114868 GB avg = 0.125 KB (by entry num) total key len = 0.025800 GB avg = 0.028 KB total ukey len = 0.018428 GB avg = 0.020 KB total ukey num = 0.000921424 Billion total entry num = 0.000921424 Billion write speed all = 0.004876400 MB/sec (with seq num) write speed all = 0.004620863 MB/sec (without seq num) 2024/11/22-16:53:09.788775 140024735041088 [compaction_job.cc:1948] [default] [JOB 5181] Generated table #215842: 921424 keys, 37601074 bytes, temperature: kWarm 2024/11/22-16:53:09.788896 140024735041088 EVENT_LOG_v1 {"time_micros": 1732265589788824, "cf_name": "default", "job": 5181, "event": "table_file_creation", "file_number": 215842, "file_size": 37601074, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702550, "largest_seqno": 3397767016, "table_properties": {"data_size": 31636736, "index_size": 1764672, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25799872, "raw_average_key_size": 28, "raw_value_size": 108417992, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921424, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265584, "oldest_key_time": 0, "file_creation_time": 1732265584, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215842, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:53:09.839962 140024735041088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215843.sst) /dev/shm/ToplingTemp/Topling-207718-J0Jn9P 2024/11/22-16:53:11.396240 140024735041088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215843.sst): first pass time = 1.56's, 86.245'MB/sec 2024/11/22-16:53:13.304111 140024735041088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215843.sst): old prealloc_size = 147639500, real_size = 33852096 2024/11/22-16:53:13.304716 140024735041088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215843.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921499 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0319 GB avg-val = 117.65 avg-zval = 34.58 2024/11/22-16:53:13.305492 140024735041088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215843.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:53:14.598465 140024735041088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215843.sst): second pass time = 1.91's, 56.847'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.29's, 29.221'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3092.607'MB/sec (index lex order gen) rebuild zvType time = 1.04's, 0.222'MB/sec write SST data time = 0.26's, 147.622'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 28130.354'MB/sec zip my value time = 1.91's, unzip length = 0.108'GB zip my value throughput = 56.847'MB/sec zip pipeline throughput = 108.950'MB/sec entries = 921499 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.65 avg-zval = 34.58 usrkeys = 921499 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.65 avg-zval = 34.58 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921499 bytes = 3110080 } seq expand size = 6450493 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.5140 value = 3.4020 dict = 1.00 all = 3.5487 } Zip/UnZip{ index = 0.0951 value = 0.2939 dict = 1.00 all = 0.2818 } ---------------------------- total value len = 0.229734 GB avg = 0.125 KB (by entry num) total key len = 0.051602 GB avg = 0.028 KB total ukey len = 0.036858 GB avg = 0.020 KB total ukey num = 0.001842923 Billion total entry num = 0.001842923 Billion write speed all = 0.009751176 MB/sec (with seq num) write speed all = 0.009240167 MB/sec (without seq num) 2024/11/22-16:53:14.648586 140024735041088 [compaction_job.cc:1948] [default] [JOB 5181] Generated table #215843: 921499 keys, 37821430 bytes, temperature: kWarm 2024/11/22-16:53:14.648667 140024735041088 EVENT_LOG_v1 {"time_micros": 1732265594648624, "cf_name": "default", "job": 5181, "event": "table_file_creation", "file_number": 215843, "file_size": 37821430, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702561, "largest_seqno": 3397767950, "table_properties": {"data_size": 31868656, "index_size": 1752896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25801972, "raw_average_key_size": 28, "raw_value_size": 108415790, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921499, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265589, "oldest_key_time": 0, "file_creation_time": 1732265589, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215843, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:53:14.699305 140024735041088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215844.sst) /dev/shm/ToplingTemp/Topling-207718-mYKfPQ 2024/11/22-16:53:15.895629 140024735041088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215844.sst): first pass time = 1.20's, 112.197'MB/sec 2024/11/22-16:53:17.395984 140024735041088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215844.sst): old prealloc_size = 147639500, real_size = 33930272 2024/11/22-16:53:17.396512 140024735041088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215844.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923547 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0319 GB avg-val = 117.33 avg-zval = 34.59 2024/11/22-16:53:17.397264 140024735041088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215844.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:53:18.653923 140024735041088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215844.sst): second pass time = 1.50's, 72.254'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.26's, 30.134'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3455.887'MB/sec (index lex order gen) rebuild zvType time = 1.02's, 0.227'MB/sec write SST data time = 0.24's, 156.599'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 37584.612'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 72.254'MB/sec zip pipeline throughput = 72.254'MB/sec entries = 923547 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.33 avg-zval = 34.59 usrkeys = 923547 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.33 avg-zval = 34.59 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923547 bytes = 3116992 } seq expand size = 6464829 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.5132 value = 3.3923 dict = 1.00 all = 3.5409 } Zip/UnZip{ index = 0.0951 value = 0.2948 dict = 1.00 all = 0.2824 } ---------------------------- total value len = 0.344558 GB avg = 0.125 KB (by entry num) total key len = 0.077461 GB avg = 0.028 KB total ukey len = 0.055329 GB avg = 0.020 KB total ukey num = 0.002766470 Billion total entry num = 0.002766470 Billion write speed all = 0.014625213 MB/sec (with seq num) write speed all = 0.013858229 MB/sec (without seq num) 2024/11/22-16:53:18.690939 140024735041088 [compaction_job.cc:1948] [default] [JOB 5181] Generated table #215844: 923547 keys, 37904736 bytes, temperature: kWarm 2024/11/22-16:53:18.691020 140024735041088 EVENT_LOG_v1 {"time_micros": 1732265598690975, "cf_name": "default", "job": 5181, "event": "table_file_creation", "file_number": 215844, "file_size": 37904736, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702575, "largest_seqno": 3397767218, "table_properties": {"data_size": 31942288, "index_size": 1756928, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25859316, "raw_average_key_size": 28, "raw_value_size": 108358450, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923547, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265594, "oldest_key_time": 0, "file_creation_time": 1732265594, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215844, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:53:18.759060 140024735041088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215845.sst) /dev/shm/ToplingTemp/Topling-207718-HMFqHP 2024/11/22-16:53:19.893939 140024735041088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215845.sst): first pass time = 1.13's, 118.271'MB/sec 2024/11/22-16:53:22.123194 140024735041088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215845.sst): old prealloc_size = 147639500, real_size = 33736848 2024/11/22-16:53:22.123753 140024735041088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215845.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921319 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0318 GB avg-val = 117.68 avg-zval = 34.47 2024/11/22-16:53:22.124523 140024735041088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215845.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:53:23.284501 140024735041088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215845.sst): second pass time = 2.23's, 48.650'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.16's, 32.469'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3216.549'MB/sec (index lex order gen) rebuild zvType time = 0.93's, 0.247'MB/sec write SST data time = 0.23's, 165.364'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 35487.114'MB/sec zip my value time = 2.23's, unzip length = 0.108'GB zip my value throughput = 48.650'MB/sec zip pipeline throughput = 109.854'MB/sec entries = 921319 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.68 avg-zval = 34.47 usrkeys = 921319 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.68 avg-zval = 34.47 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921319 bytes = 3109472 } seq expand size = 6449233 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5131 value = 3.4144 dict = 1.00 all = 3.5596 } Zip/UnZip{ index = 0.0951 value = 0.2929 dict = 1.00 all = 0.2809 } ---------------------------- total value len = 0.459428 GB avg = 0.125 KB (by entry num) total key len = 0.103258 GB avg = 0.028 KB total ukey len = 0.073756 GB avg = 0.020 KB total ukey num = 0.003687789 Billion total entry num = 0.003687789 Billion write speed all = 0.019496954 MB/sec (with seq num) write speed all = 0.018474705 MB/sec (without seq num) 2024/11/22-16:53:23.325221 140024735041088 [compaction_job.cc:1948] [default] [JOB 5181] Generated table #215845: 921319 keys, 37705446 bytes, temperature: kWarm 2024/11/22-16:53:23.325297 140024735041088 EVENT_LOG_v1 {"time_micros": 1732265603325253, "cf_name": "default", "job": 5181, "event": "table_file_creation", "file_number": 215845, "file_size": 37705446, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702604, "largest_seqno": 3397767404, "table_properties": {"data_size": 31753600, "index_size": 1752704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25796932, "raw_average_key_size": 28, "raw_value_size": 108420922, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921319, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265598, "oldest_key_time": 0, "file_creation_time": 1732265598, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215845, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:53:23.394029 140024735041088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215846.sst) /dev/shm/ToplingTemp/Topling-207718-HsmWUR 2024/11/22-16:53:24.346849 140024735041088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215846.sst): first pass time = 0.95's, 140.869'MB/sec 2024/11/22-16:53:25.694595 140024735041088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215846.sst): old prealloc_size = 147639500, real_size = 33583904 2024/11/22-16:53:25.695226 140024735041088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215846.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922215 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0316 GB avg-val = 117.54 avg-zval = 34.26 2024/11/22-16:53:25.695998 140024735041088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215846.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:53:26.734694 140024735041088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215846.sst): second pass time = 1.35's, 80.511'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.04's, 36.108'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2891.882'MB/sec (index lex order gen) rebuild zvType time = 0.89's, 0.260'MB/sec write SST data time = 0.15's, 246.680'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 35665.602'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.511'MB/sec zip pipeline throughput = 80.511'MB/sec entries = 922215 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.54 avg-zval = 34.26 usrkeys = 922215 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.54 avg-zval = 34.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922215 bytes = 3112496 } seq expand size = 6455505 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5126 value = 3.4304 dict = 1.00 all = 3.5739 } Zip/UnZip{ index = 0.0951 value = 0.2915 dict = 1.00 all = 0.2798 } ---------------------------- total value len = 0.574279 GB avg = 0.125 KB (by entry num) total key len = 0.129080 GB avg = 0.028 KB total ukey len = 0.092200 GB avg = 0.020 KB total ukey num = 0.004610004 Billion total entry num = 0.004610004 Billion write speed all = 0.024368345 MB/sec (with seq num) write speed all = 0.023090612 MB/sec (without seq num) 2024/11/22-16:53:26.770568 140024735041088 [compaction_job.cc:1948] [default] [JOB 5181] Generated table #215846: 922215 keys, 37555260 bytes, temperature: kWarm 2024/11/22-16:53:26.770625 140024735041088 EVENT_LOG_v1 {"time_micros": 1732265606770594, "cf_name": "default", "job": 5181, "event": "table_file_creation", "file_number": 215846, "file_size": 37555260, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702617, "largest_seqno": 3397768208, "table_properties": {"data_size": 31598672, "index_size": 1754496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25822020, "raw_average_key_size": 28, "raw_value_size": 108395810, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922215, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265603, "oldest_key_time": 0, "file_creation_time": 1732265603, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215846, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:53:26.809195 140024735041088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215847.sst) /dev/shm/ToplingTemp/Topling-207718-8ml5vS 2024/11/22-16:53:27.616608 140024735041088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215847.sst): first pass time = 0.81's, 166.239'MB/sec 2024/11/22-16:53:29.197160 140024735041088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215847.sst): old prealloc_size = 147639500, real_size = 33720416 2024/11/22-16:53:29.197716 140024735041088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215847.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922718 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0317 GB avg-val = 117.46 avg-zval = 34.39 2024/11/22-16:53:29.198468 140024735041088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215847.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:53:30.414325 140024735041088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215847.sst): second pass time = 1.58's, 68.638'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.22's, 30.968'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3192.947'MB/sec (index lex order gen) rebuild zvType time = 1.07's, 0.216'MB/sec write SST data time = 0.15's, 254.201'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 36057.993'MB/sec zip my value time = 1.58's, unzip length = 0.108'GB zip my value throughput = 68.638'MB/sec zip pipeline throughput = 68.638'MB/sec entries = 922718 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.46 avg-zval = 34.39 usrkeys = 922718 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.46 avg-zval = 34.39 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922718 bytes = 3114192 } seq expand size = 6459026 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5110 value = 3.4153 dict = 1.00 all = 3.5608 } Zip/UnZip{ index = 0.0951 value = 0.2928 dict = 1.00 all = 0.2808 } ---------------------------- total value len = 0.689120 GB avg = 0.125 KB (by entry num) total key len = 0.154916 GB avg = 0.028 KB total ukey len = 0.110654 GB avg = 0.020 KB total ukey num = 0.005532722 Billion total entry num = 0.005532722 Billion write speed all = 0.029238458 MB/sec (with seq num) write speed all = 0.027705175 MB/sec (without seq num) 2024/11/22-16:53:30.450192 140024735041088 [compaction_job.cc:1948] [default] [JOB 5181] Generated table #215847: 922718 keys, 37692854 bytes, temperature: kWarm 2024/11/22-16:53:30.450253 140024735041088 EVENT_LOG_v1 {"time_micros": 1732265610450221, "cf_name": "default", "job": 5181, "event": "table_file_creation", "file_number": 215847, "file_size": 37692854, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702634, "largest_seqno": 3397767845, "table_properties": {"data_size": 31733840, "index_size": 1755712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25836104, "raw_average_key_size": 28, "raw_value_size": 108381648, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922718, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265606, "oldest_key_time": 0, "file_creation_time": 1732265606, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215847, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:53:30.487341 140024735041088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215848.sst) /dev/shm/ToplingTemp/Topling-207718-DSrSTS 2024/11/22-16:53:31.659396 140024735041088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215848.sst): first pass time = 1.17's, 114.518'MB/sec 2024/11/22-16:53:33.068441 140024735041088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215848.sst): old prealloc_size = 147639500, real_size = 33927520 2024/11/22-16:53:33.069036 140024735041088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215848.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921384 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0319 GB avg-val = 117.67 avg-zval = 34.66 2024/11/22-16:53:33.069790 140024735041088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215848.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:53:34.092720 140024735041088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215848.sst): second pass time = 1.40's, 77.575'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.02's, 36.999'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3092.697'MB/sec (index lex order gen) rebuild zvType time = 0.90's, 0.257'MB/sec write SST data time = 0.13's, 299.080'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 32376.333'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.575'MB/sec zip pipeline throughput = 77.575'MB/sec entries = 921384 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.67 avg-zval = 34.66 usrkeys = 921384 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.67 avg-zval = 34.66 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921384 bytes = 3109680 } seq expand size = 6449688 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.4422 value = 3.3953 dict = 1.00 all = 3.5417 } Zip/UnZip{ index = 0.0958 value = 0.2945 dict = 1.00 all = 0.2824 } ---------------------------- total value len = 0.803988 GB avg = 0.125 KB (by entry num) total key len = 0.180715 GB avg = 0.028 KB total ukey len = 0.129082 GB avg = 0.020 KB total ukey num = 0.006454106 Billion total entry num = 0.006454106 Billion write speed all = 0.034107010 MB/sec (with seq num) write speed all = 0.032318611 MB/sec (without seq num) 2024/11/22-16:53:34.127106 140024735041088 [compaction_job.cc:1948] [default] [JOB 5181] Generated table #215848: 921384 keys, 37896488 bytes, temperature: kWarm 2024/11/22-16:53:34.127173 140024735041088 EVENT_LOG_v1 {"time_micros": 1732265614127138, "cf_name": "default", "job": 5181, "event": "table_file_creation", "file_number": 215848, "file_size": 37896488, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702661, "largest_seqno": 3397767101, "table_properties": {"data_size": 31932240, "index_size": 1764736, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25798752, "raw_average_key_size": 28, "raw_value_size": 108419040, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921384, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265610, "oldest_key_time": 0, "file_creation_time": 1732265610, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215848, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:53:34.162738 140024735041088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215849.sst) /dev/shm/ToplingTemp/Topling-207718-pHC4bP 2024/11/22-16:53:35.018848 140024735041088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215849.sst): first pass time = 0.86's, 156.784'MB/sec 2024/11/22-16:53:36.444502 140024735041088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215849.sst): old prealloc_size = 147639500, real_size = 36788656 2024/11/22-16:53:36.445096 140024735041088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215849.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923857 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0348 GB avg-val = 117.28 avg-zval = 37.67 2024/11/22-16:53:36.445870 140024735041088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215849.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:53:37.394892 140024735041088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215849.sst): second pass time = 1.42's, 76.068'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 43.018'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3074.137'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.281'MB/sec write SST data time = 0.13's, 320.979'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 31690.880'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 76.068'MB/sec zip pipeline throughput = 76.068'MB/sec entries = 923857 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.28 avg-zval = 37.67 usrkeys = 923857 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.28 avg-zval = 37.67 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923857 bytes = 3233520 } seq expand size = 6466999 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0348 GB dict = 1.08 MB all = 0.0409 GB } UnZip/Zip{ index = 10.5121 value = 3.1135 dict = 1.00 all = 3.2830 } Zip/UnZip{ index = 0.0951 value = 0.3212 dict = 1.00 all = 0.3046 } ---------------------------- total value len = 0.918805 GB avg = 0.125 KB (by entry num) total key len = 0.206583 GB avg = 0.028 KB total ukey len = 0.147559 GB avg = 0.020 KB total ukey num = 0.007377963 Billion total entry num = 0.007377963 Billion write speed all = 0.038975427 MB/sec (with seq num) write speed all = 0.036931266 MB/sec (without seq num) 2024/11/22-16:53:37.437345 140024735041088 [compaction_job.cc:1948] [default] [JOB 5181] Generated table #215849: 923857 keys, 40883186 bytes, temperature: kWarm 2024/11/22-16:53:37.437408 140024735041088 EVENT_LOG_v1 {"time_micros": 1732265617437375, "cf_name": "default", "job": 5181, "event": "table_file_creation", "file_number": 215849, "file_size": 40883186, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868609, "largest_seqno": 3397766841, "table_properties": {"data_size": 34799840, "index_size": 1757696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25867996, "raw_average_key_size": 28, "raw_value_size": 108349770, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923857, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265614, "oldest_key_time": 0, "file_creation_time": 1732265614, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215849, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:53:37.481266 140024735041088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215850.sst) /dev/shm/ToplingTemp/Topling-207718-wYTkCT 2024/11/22-16:53:38.321958 140024735041088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215850.sst): first pass time = 0.84's, 159.660'MB/sec 2024/11/22-16:53:39.719496 140024735041088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215850.sst): old prealloc_size = 147639500, real_size = 36625216 2024/11/22-16:53:39.720088 140024735041088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215850.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922581 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0346 GB avg-val = 117.48 avg-zval = 37.55 2024/11/22-16:53:39.720816 140024735041088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215850.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:53:40.653055 140024735041088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215850.sst): second pass time = 1.40's, 77.596'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 43.345'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3078.736'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.288'MB/sec write SST data time = 0.13's, 307.980'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32697.563'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.596'MB/sec zip pipeline throughput = 77.596'MB/sec entries = 922581 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.48 avg-zval = 37.55 usrkeys = 922581 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.10 avg-zval = 37.55 TagRS{ kinds = RS_Key0_TagN bytes = 144240 } TagArray{ size = 872798 bytes = 2836608 } seq expand size = 6109586 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0346 GB dict = 1.08 MB all = 0.0405 GB } UnZip/Zip{ index = 10.5141 value = 3.1290 dict = 1.00 all = 3.3169 } Zip/UnZip{ index = 0.0951 value = 0.3196 dict = 1.00 all = 0.3015 } ---------------------------- total value len = 1.033300 GB avg = 0.124 KB (by entry num) total key len = 0.232415 GB avg = 0.028 KB total ukey len = 0.166011 GB avg = 0.020 KB total ukey num = 0.008300544 Billion total entry num = 0.008300544 Billion write speed all = 0.043830421 MB/sec (with seq num) write speed all = 0.041530906 MB/sec (without seq num) 2024/11/22-16:53:40.690840 140024735041088 [compaction_job.cc:1948] [default] [JOB 5181] Generated table #215850: 922581 keys, 40464240 bytes, temperature: kWarm 2024/11/22-16:53:40.690902 140024735041088 EVENT_LOG_v1 {"time_micros": 1732265620690870, "cf_name": "default", "job": 5181, "event": "table_file_creation", "file_number": 215850, "file_size": 40464240, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700489, "table_properties": {"data_size": 34639472, "index_size": 1754944, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25832268, "raw_average_key_size": 28, "raw_value_size": 108385490, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922581, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265617, "oldest_key_time": 0, "file_creation_time": 1732265617, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215850, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:53:40.728387 140024735041088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215851.sst) /dev/shm/ToplingTemp/Topling-207718-fAu61P 2024/11/22-16:53:41.581377 140024735041088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215851.sst): first pass time = 0.85's, 157.357'MB/sec 2024/11/22-16:53:42.991776 140024735041088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215851.sst): old prealloc_size = 147639500, real_size = 36589616 2024/11/22-16:53:42.992337 140024735041088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215851.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924029 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0346 GB avg-val = 117.25 avg-zval = 37.46 2024/11/22-16:53:42.992881 140024735041088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215851.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:53:43.867390 140024735041088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215851.sst): second pass time = 1.41's, 76.856'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 45.162'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3281.351'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.310'MB/sec write SST data time = 0.13's, 303.406'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 30610.145'MB/sec zip my value time = 1.41's, unzip length = 0.108'GB zip my value throughput = 76.856'MB/sec zip pipeline throughput = 76.856'MB/sec entries = 924029 avg-key = 28.00 avg-zkey = 1.89 avg-val = 117.25 avg-zval = 37.46 usrkeys = 924029 avg-key = 20.00 avg-zkey = 1.89 avg-val = 121.80 avg-zval = 37.46 TagRS{ kinds = RS_Key0_TagN bytes = 144480 } TagArray{ size = 599820 bytes = 1949424 } seq expand size = 4198740 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0346 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.5831 value = 3.1303 dict = 1.00 all = 3.3941 } Zip/UnZip{ index = 0.0945 value = 0.3195 dict = 1.00 all = 0.2946 } ---------------------------- total value len = 1.145844 GB avg = 0.124 KB (by entry num) total key len = 0.258288 GB avg = 0.028 KB total ukey len = 0.184491 GB avg = 0.020 KB total ukey num = 0.009224573 Billion total entry num = 0.009224573 Billion write speed all = 0.048618231 MB/sec (with seq num) write speed all = 0.046063016 MB/sec (without seq num) 2024/11/22-16:53:43.906393 140024735041088 [compaction_job.cc:1948] [default] [JOB 5181] Generated table #215851: 924029 keys, 39544133 bytes, temperature: kWarm 2024/11/22-16:53:43.906457 140024735041088 EVENT_LOG_v1 {"time_micros": 1732265623906425, "cf_name": "default", "job": 5181, "event": "table_file_creation", "file_number": 215851, "file_size": 39544133, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700833, "table_properties": {"data_size": 34612192, "index_size": 1746240, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25872812, "raw_average_key_size": 28, "raw_value_size": 108344994, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924029, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265620, "oldest_key_time": 0, "file_creation_time": 1732265620, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215851, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:53:43.945981 140024735041088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215852.sst) /dev/shm/ToplingTemp/Topling-207718-agqa9S 2024/11/22-16:53:44.714883 140024735041088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215852.sst): first pass time = 0.77's, 174.566'MB/sec 2024/11/22-16:53:46.346554 140024735041088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215852.sst): old prealloc_size = 147639500, real_size = 36491632 2024/11/22-16:53:46.347091 140024735041088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215852.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927142 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.77 avg-zval = 37.21 2024/11/22-16:53:46.347135 140024735041088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215852.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:53:47.099796 140024735041088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215852.sst): second pass time = 1.63's, 66.394'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.75's, 49.587'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3431.571'MB/sec (index lex order gen) rebuild zvType time = 0.63's, 0.370'MB/sec write SST data time = 0.13's, 296.719'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 24402.755'MB/sec zip my value time = 1.63's, unzip length = 0.108'GB zip my value throughput = 66.394'MB/sec zip pipeline throughput = 66.394'MB/sec entries = 927142 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.77 avg-zval = 37.21 usrkeys = 927142 avg-key = 20.00 avg-zkey = 1.90 avg-val = 116.77 avg-zval = 37.21 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5131 value = 3.1383 dict = 1.00 all = 3.5935 } Zip/UnZip{ index = 0.0951 value = 0.3186 dict = 1.00 all = 0.2783 } ---------------------------- total value len = 1.254102 GB avg = 0.124 KB (by entry num) total key len = 0.284248 GB avg = 0.028 KB total ukey len = 0.203034 GB avg = 0.020 KB total ukey num = 0.010151715 Billion total entry num = 0.010151715 Billion write speed all = 0.053259576 MB/sec (with seq num) write speed all = 0.050447856 MB/sec (without seq num) 2024/11/22-16:53:47.133533 140024735041088 [compaction_job.cc:1948] [default] [JOB 5181] Generated table #215852: 927142 keys, 37349996 bytes, temperature: kWarm 2024/11/22-16:53:47.133593 140024735041088 EVENT_LOG_v1 {"time_micros": 1732265627133562, "cf_name": "default", "job": 5181, "event": "table_file_creation", "file_number": 215852, "file_size": 37349996, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34495904, "index_size": 1763776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25959976, "raw_average_key_size": 28, "raw_value_size": 108257792, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927142, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265623, "oldest_key_time": 0, "file_creation_time": 1732265623, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215852, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:53:47.169793 140024735041088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215853.sst) /dev/shm/ToplingTemp/Topling-207718-3ZMuIS 2024/11/22-16:53:47.935954 140024735041088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215853.sst): first pass time = 0.77's, 175.190'MB/sec 2024/11/22-16:53:49.535144 140024735041088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215853.sst): old prealloc_size = 147639500, real_size = 36369936 2024/11/22-16:53:49.535725 140024735041088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215853.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926829 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0344 GB avg-val = 116.81 avg-zval = 37.10 2024/11/22-16:53:49.535791 140024735041088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215853.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:53:50.314946 140024735041088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215853.sst): second pass time = 1.60's, 67.749'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.78's, 47.741'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3135.743'MB/sec (index lex order gen) rebuild zvType time = 0.63's, 0.365'MB/sec write SST data time = 0.14's, 257.012'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33955.121'MB/sec zip my value time = 1.60's, unzip length = 0.108'GB zip my value throughput = 67.749'MB/sec zip pipeline throughput = 67.749'MB/sec entries = 926829 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.81 avg-zval = 37.10 usrkeys = 926829 avg-key = 20.00 avg-zkey = 1.89 avg-val = 116.81 avg-zval = 37.10 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5833 value = 3.1485 dict = 1.00 all = 3.6053 } Zip/UnZip{ index = 0.0945 value = 0.3176 dict = 1.00 all = 0.2774 } ---------------------------- total value len = 1.362368 GB avg = 0.123 KB (by entry num) total key len = 0.310199 GB avg = 0.028 KB total ukey len = 0.221571 GB avg = 0.020 KB total ukey num = 0.011078544 Billion total entry num = 0.011078544 Billion write speed all = 0.057899917 MB/sec (with seq num) write speed all = 0.054831835 MB/sec (without seq num) 2024/11/22-16:53:50.347813 140024735041088 [compaction_job.cc:1948] [default] [JOB 5181] Generated table #215853: 926829 keys, 37228132 bytes, temperature: kWarm 2024/11/22-16:53:50.347871 140024735041088 EVENT_LOG_v1 {"time_micros": 1732265630347841, "cf_name": "default", "job": 5181, "event": "table_file_creation", "file_number": 215853, "file_size": 37228132, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34386560, "index_size": 1751488, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25951212, "raw_average_key_size": 28, "raw_value_size": 108266582, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926829, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265627, "oldest_key_time": 0, "file_creation_time": 1732265627, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215853, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:53:50.383676 140024735041088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215854.sst) /dev/shm/ToplingTemp/Topling-207718-wWRJUP 2024/11/22-16:53:51.185391 140024735041088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215854.sst): first pass time = 0.80's, 167.420'MB/sec 2024/11/22-16:53:52.570805 140024735041088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215854.sst): old prealloc_size = 147639500, real_size = 34737328 2024/11/22-16:53:52.571363 140024735041088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215854.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924061 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0328 GB avg-val = 117.25 avg-zval = 35.45 2024/11/22-16:53:52.572702 140024735041088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215854.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:53:53.580917 140024735041088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215854.sst): second pass time = 1.38's, 78.270'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.01's, 38.334'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3256.173'MB/sec (index lex order gen) rebuild zvType time = 0.88's, 0.263'MB/sec write SST data time = 0.13's, 295.289'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 30448.234'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.270'MB/sec zip pipeline throughput = 78.270'MB/sec entries = 924061 avg-key = 28.00 avg-zkey = 1.89 avg-val = 117.25 avg-zval = 35.45 usrkeys = 924061 avg-key = 20.00 avg-zkey = 1.89 avg-val = 123.94 avg-zval = 35.45 TagRS{ kinds = RS_Key0_TagN bytes = 144480 } TagArray{ size = 882780 bytes = 2979392 } seq expand size = 6179460 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0328 GB dict = 1.08 MB all = 0.0387 GB } UnZip/Zip{ index = 10.5811 value = 3.3073 dict = 1.00 all = 3.4662 } Zip/UnZip{ index = 0.0945 value = 0.3024 dict = 1.00 all = 0.2885 } ---------------------------- total value len = 1.476892 GB avg = 0.123 KB (by entry num) total key len = 0.336073 GB avg = 0.028 KB total ukey len = 0.240052 GB avg = 0.020 KB total ukey num = 0.012002605 Billion total entry num = 0.012002605 Billion write speed all = 0.062753007 MB/sec (with seq num) write speed all = 0.059429393 MB/sec (without seq num) 2024/11/22-16:53:53.616072 140024735041088 [compaction_job.cc:1948] [default] [JOB 5181] Generated table #215854: 924061 keys, 38721738 bytes, temperature: kWarm 2024/11/22-16:53:53.616135 140024735041088 EVENT_LOG_v1 {"time_micros": 1732265633616102, "cf_name": "default", "job": 5181, "event": "table_file_creation", "file_number": 215854, "file_size": 38721738, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3397768052, "table_properties": {"data_size": 32759520, "index_size": 1746624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25873708, "raw_average_key_size": 28, "raw_value_size": 108344058, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924061, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265630, "oldest_key_time": 0, "file_creation_time": 1732265630, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215854, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:53:53.654294 140024735041088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215855.sst) /dev/shm/ToplingTemp/Topling-207718-9wBDOR 2024/11/22-16:53:54.476838 140024735041088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215855.sst): first pass time = 0.82's, 163.181'MB/sec 2024/11/22-16:53:55.895760 140024735041088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215855.sst): old prealloc_size = 147639500, real_size = 35157872 2024/11/22-16:53:55.896347 140024735041088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215855.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925809 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0332 GB avg-val = 116.97 avg-zval = 35.81 2024/11/22-16:53:55.897638 140024735041088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215855.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:53:56.796734 140024735041088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215855.sst): second pass time = 1.42's, 76.383'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 43.439'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3207.624'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.303'MB/sec write SST data time = 0.14's, 287.343'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 39085.863'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 76.383'MB/sec zip pipeline throughput = 87.333'MB/sec entries = 925809 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.97 avg-zval = 35.81 usrkeys = 925809 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.97 avg-zval = 35.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925809 bytes = 3124624 } seq expand size = 6480663 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0391 GB } UnZip/Zip{ index = 10.4423 value = 3.2665 dict = 1.00 all = 3.4294 } Zip/UnZip{ index = 0.0958 value = 0.3061 dict = 1.00 all = 0.2916 } ---------------------------- total value len = 1.591668 GB avg = 0.123 KB (by entry num) total key len = 0.361996 GB avg = 0.028 KB total ukey len = 0.258568 GB avg = 0.020 KB total ukey num = 0.012928414 Billion total entry num = 0.012928414 Billion write speed all = 0.067615544 MB/sec (with seq num) write speed all = 0.064035964 MB/sec (without seq num) 2024/11/22-16:53:56.806038 140024735041088 [compaction_job.cc:1948] [default] [JOB 5181] Generated table #215855: 925809 keys, 39137064 bytes, temperature: kWarm 2024/11/22-16:53:56.806095 140024735041088 EVENT_LOG_v1 {"time_micros": 1732265636806063, "cf_name": "default", "job": 5181, "event": "table_file_creation", "file_number": 215855, "file_size": 39137064, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701386, "largest_seqno": 3397768437, "table_properties": {"data_size": 33153056, "index_size": 1773184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25922652, "raw_average_key_size": 28, "raw_value_size": 108295174, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925809, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265633, "oldest_key_time": 0, "file_creation_time": 1732265633, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215855, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:53:56.843437 140024735041088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215856.sst) /dev/shm/ToplingTemp/Topling-207718-Kdj9dQ 2024/11/22-16:53:56.920326 140024735041088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215856.sst): first pass time = 0.08's, 229.872'MB/sec 2024/11/22-16:53:57.075461 140024735041088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215856.sst): old prealloc_size = 147639500, real_size = 5068368 2024/11/22-16:53:57.075581 140024735041088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215856.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 122932 prefix = 4 raw-key = 0.0020 GB zip-key = 0.0002 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0142 GB zip-val = 0.0048 GB avg-val = 115.71 avg-zval = 39.05 2024/11/22-16:53:57.075703 140024735041088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215856.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.018 GB 2024/11/22-16:53:57.193142 140024735041088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00/215856.sst): second pass time = 0.15's, 91.993'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.12's, 47.224'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2410.200'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.306'MB/sec write SST data time = 0.02's, 321.880'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.149'MB, throughput = 5695.775'MB/sec zip my value time = 0.15's, unzip length = 0.014'GB zip my value throughput = 91.993'MB/sec zip pipeline throughput = 91.993'MB/sec entries = 122932 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.71 avg-zval = 39.05 usrkeys = 122932 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.71 avg-zval = 39.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 122932 bytes = 368816 } seq expand size = 860524 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0025 GB value = 0.0142 GB dict = 0.15 MB all = 0.0177 GB } __ZipSize{ index = 0.0002 GB value = 0.0048 GB dict = 0.15 MB all = 0.0056 GB } UnZip/Zip{ index = 10.3884 value = 2.9628 dict = 1.00 all = 3.1792 } Zip/UnZip{ index = 0.0963 value = 0.3375 dict = 1.00 all = 0.3145 } ---------------------------- total value len = 1.606752 GB avg = 0.123 KB (by entry num) total key len = 0.365438 GB avg = 0.028 KB total ukey len = 0.261027 GB avg = 0.020 KB total ukey num = 0.013051346 Billion total entry num = 0.013051346 Billion write speed all = 0.068255805 MB/sec (with seq num) write speed all = 0.064642237 MB/sec (without seq num) 2024/11/22-16:53:57.204631 140024735041088 [compaction_job.cc:1948] [default] [JOB 5181] Generated table #215856: 122932 keys, 5556793 bytes, temperature: kWarm 2024/11/22-16:53:57.204682 140024735041088 EVENT_LOG_v1 {"time_micros": 1732265637204654, "cf_name": "default", "job": 5181, "event": "table_file_creation", "file_number": 215856, "file_size": 5556793, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3382133338, "largest_seqno": 3396618501, "table_properties": {"data_size": 4800768, "index_size": 236672, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3442096, "raw_average_key_size": 28, "raw_value_size": 14223928, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 122932, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265636, "oldest_key_time": 0, "file_creation_time": 1732265636, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215856, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:53:57.217475 140024735041088 [compaction_job.cc:703] job-05181: subcompact[0], size: 0.543063 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05181/att-00 [215842,215843,215844,215845,215846,215847,215848,215849,215850,215851,215852,215853,215854,215855,215856] 2024/11/22-16:53:57.311490 140024735041088 (Original Log Time 2024/11/22-16:53:04.082528) EVENT_LOG_v1 {"time_micros": 1732265584082489, "job": 5181, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [215825], "files_L3": [215775, 215776, 215777, 215778, 215779, 215780, 215781, 215782, 215783, 215784, 215785, 215786, 215787, 215794], "score": 0.900781, "input_data_size": 566432270, "oldest_snapshot_seqno": 3397818394} 2024/11/22-16:53:57.311494 140024735041088 (Original Log Time 2024/11/22-16:53:57.310724) [compaction_job.cc:2022] [default] [JOB 5181] Compacted 1@2 + 14@3 files to L3 => 543062570 bytes 2024/11/22-16:53:57.311495 140024735041088 (Original Log Time 2024/11/22-16:53:57.311428) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 10.7 rd, 10.2 wr, level 3, files in(1, 14) out(15 +0 blob) MB in(27.1, 513.1 +0.0 blob) out(517.9 +0.0 blob), read-write-amplify(39.0) write-amplify(19.1) OK, records in: 13162896, records dropped: 111550 output_compression: Snappy 2024/11/22-16:53:57.311501 140024735041088 (Original Log Time 2024/11/22-16:53:57.311471) EVENT_LOG_v1 {"time_micros": 1732265637311451, "cf": "default", "job": 5181, "event": "compaction_finished", "compaction_time_micros": 53134953, "compaction_time_cpu_micros": 23343876, "output_level": 3, "num_output_files": 15, "total_output_size": 543062570, "num_input_records": 13162896, "num_output_records": 13051346, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 376146067, "file_range_sync_nanos": 0, "file_fsync_nanos": 14803, "file_prepare_write_nanos": 33996, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/22-16:53:57.433061 140024735041088 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732265583,"executesMs":53385,"compactionJobId":5181,"attempt":0,"compactionInputRawBytes":1912555228,"compactionInputZipBytes":566432270,"compactionOutputRawBytes":0,"compactionOutputZipBytes":543062570,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-7e7112a2cc99a82937be31f09ef05d5a-36273164b4af307a-00"}}, response = 2024/11/22-16:53:57.433128 140024735041088 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-05181/att-00: olev 3, work 53.386 s, result 6.311 ms, install 0.800 ms, input{raw 1.781 GiB zip 540.192 MiB}