2024/11/23-05:19:40.520907 140024760219200 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00: fork to child time = 0.009909 sec 2024/11/23-05:19:40.521725 140024760219200 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00 2024/11/23-05:19:40.530112 140024760219200 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00 2024/11/23-05:19:40.531218 140024760219200 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-05:19:40.531485 140024760219200 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-05:19:40.583277 140024760219200 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-20035/att-00/MANIFEST-206347 2024/11/23-05:19:40.597659 140024760219200 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-20035/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 246335, last_sequence is 3807189893, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 246186 2024/11/23-05:19:40.597667 140024760219200 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-05:19:40.597811 140024760219200 [version_set.cc:5721] Creating manifest 246335 2024/11/23-05:19:40.636510 140024760219200 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-05:19:40.658244 140024760219200 [compaction_job.cc:2437] [default] [JOB 20035] Compacting 1@2 + 15@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-05:19:40.658263 140024760219200 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 14984 Base level 2, inputs: [246328(27MB)], [246269(35MB) 246270(35MB) 246271(34MB) 246272(34MB) 246273(36MB) 246274(33MB) 246275(33MB) 246276(33MB) 246277(35MB) 246278(35MB) 246279(37MB) 246280(35MB) 246281(35MB) 246282(34MB) 246290(35MB)] 2024/11/23-05:19:40.664298 140024760219200 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-05:19:40.664331 140024760219200 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-05:19:40.664351 140024760219200 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-05:19:40.664356 140024760219200 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-05:19:40.664367 140024760219200 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-05:19:41.229808 140024760219200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246336.sst) /dev/shm/ToplingTemp/Topling-419589-WFGL9y 2024/11/23-05:19:41.894109 140024760219200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246336.sst): first pass time = 0.66's, 202.058'MB/sec 2024/11/23-05:19:43.818965 140024760219200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246336.sst): old prealloc_size = 147639500, real_size = 32873568 2024/11/23-05:19:43.819649 140024760219200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246336.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927435 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.72 avg-zval = 33.29 2024/11/23-05:19:43.820505 140024760219200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246336.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:19:45.330707 140024760219200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246336.sst): second pass time = 1.92's, 56.260'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.51's, 24.381'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2645.361'MB/sec (index lex order gen) rebuild zvType time = 1.22's, 0.190'MB/sec write SST data time = 0.29's, 126.232'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34680.258'MB/sec zip my value time = 1.92's, unzip length = 0.108'GB zip my value throughput = 56.260'MB/sec zip pipeline throughput = 78.144'MB/sec entries = 927435 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.72 avg-zval = 33.29 usrkeys = 927435 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.72 avg-zval = 33.29 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927435 bytes = 3130112 } seq expand size = 6492045 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5104 value = 3.5059 dict = 1.00 all = 3.6415 } Zip/UnZip{ index = 0.0951 value = 0.2852 dict = 1.00 all = 0.2746 } ---------------------------- total value len = 0.114742 GB avg = 0.124 KB (by entry num) total key len = 0.025968 GB avg = 0.028 KB total ukey len = 0.018549 GB avg = 0.020 KB total ukey num = 0.000927435 Billion total entry num = 0.000927435 Billion write speed all = 0.001910721 MB/sec (with seq num) write speed all = 0.001809971 MB/sec (without seq num) 2024/11/23-05:19:45.367041 140024760219200 [compaction_job.cc:1948] [default] [JOB 20035] Generated table #246336: 927435 keys, 36857866 bytes, temperature: kWarm 2024/11/23-05:19:45.367170 140024760219200 EVENT_LOG_v1 {"time_micros": 1732310385367083, "cf_name": "default", "job": 20035, "event": "table_file_creation", "file_number": 246336, "file_size": 36857866, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3726348054, "largest_seqno": 3807171583, "table_properties": {"data_size": 30876752, "index_size": 1764800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25968180, "raw_average_key_size": 28, "raw_value_size": 108249550, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927435, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732310381, "oldest_key_time": 0, "file_creation_time": 1732310381, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 246336, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:19:45.439697 140024760219200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246337.sst) /dev/shm/ToplingTemp/Topling-419589-5JnaFx 2024/11/23-05:19:46.949854 140024760219200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246337.sst): first pass time = 1.51's, 88.879'MB/sec 2024/11/23-05:19:49.371582 140024760219200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246337.sst): old prealloc_size = 147639500, real_size = 33329536 2024/11/23-05:19:49.372198 140024760219200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246337.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927638 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0313 GB avg-val = 116.69 avg-zval = 33.76 2024/11/23-05:19:49.373008 140024760219200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246337.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:19:50.896829 140024760219200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246337.sst): second pass time = 2.42's, 44.715'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.53's, 24.617'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2965.351'MB/sec (index lex order gen) rebuild zvType time = 1.29's, 0.180'MB/sec write SST data time = 0.23's, 160.866'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32977.103'MB/sec zip my value time = 2.42's, unzip length = 0.108'GB zip my value throughput = 44.715'MB/sec zip pipeline throughput = 125.263'MB/sec entries = 927638 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.69 avg-zval = 33.76 usrkeys = 927638 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.69 avg-zval = 33.76 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927638 bytes = 3362704 } seq expand size = 6493466 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4452 value = 3.4559 dict = 1.00 all = 3.5747 } Zip/UnZip{ index = 0.0957 value = 0.2894 dict = 1.00 all = 0.2797 } ---------------------------- total value len = 0.229479 GB avg = 0.124 KB (by entry num) total key len = 0.051942 GB avg = 0.028 KB total ukey len = 0.037101 GB avg = 0.020 KB total ukey num = 0.001855073 Billion total entry num = 0.001855073 Billion write speed all = 0.003821173 MB/sec (with seq num) write speed all = 0.003619665 MB/sec (without seq num) 2024/11/23-05:19:50.935632 140024760219200 [compaction_job.cc:1948] [default] [JOB 20035] Generated table #246337: 927638 keys, 37546454 bytes, temperature: kWarm 2024/11/23-05:19:50.935714 140024760219200 EVENT_LOG_v1 {"time_micros": 1732310390935669, "cf_name": "default", "job": 20035, "event": "table_file_creation", "file_number": 246337, "file_size": 37546454, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038037, "largest_seqno": 3807171603, "table_properties": {"data_size": 31321264, "index_size": 1776192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25973864, "raw_average_key_size": 28, "raw_value_size": 108243880, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927638, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732310385, "oldest_key_time": 0, "file_creation_time": 1732310385, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 246337, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:19:50.985030 140024760219200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246338.sst) /dev/shm/ToplingTemp/Topling-419589-uPzRfx 2024/11/23-05:19:51.949238 140024760219200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246338.sst): first pass time = 0.96's, 139.209'MB/sec 2024/11/23-05:19:54.210059 140024760219200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246338.sst): old prealloc_size = 147639500, real_size = 32742528 2024/11/23-05:19:54.210638 140024760219200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246338.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931569 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0307 GB avg-val = 116.08 avg-zval = 33.00 2024/11/23-05:19:54.211384 140024760219200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246338.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:19:55.478273 140024760219200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246338.sst): second pass time = 2.26's, 47.857'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.27's, 28.876'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3179.624'MB/sec (index lex order gen) rebuild zvType time = 1.12's, 0.208'MB/sec write SST data time = 0.15's, 250.451'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31063.659'MB/sec zip my value time = 2.26's, unzip length = 0.108'GB zip my value throughput = 47.857'MB/sec zip pipeline throughput = 106.113'MB/sec entries = 931569 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.08 avg-zval = 33.00 usrkeys = 931569 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.08 avg-zval = 33.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931569 bytes = 3027616 } seq expand size = 6520983 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5134 value = 3.5180 dict = 1.00 all = 3.6650 } Zip/UnZip{ index = 0.0951 value = 0.2843 dict = 1.00 all = 0.2729 } ---------------------------- total value len = 0.344134 GB avg = 0.123 KB (by entry num) total key len = 0.078026 GB avg = 0.028 KB total ukey len = 0.055733 GB avg = 0.020 KB total ukey num = 0.002786642 Billion total entry num = 0.002786642 Billion write speed all = 0.005731786 MB/sec (with seq num) write speed all = 0.005429106 MB/sec (without seq num) 2024/11/23-05:19:55.516441 140024760219200 [compaction_job.cc:1948] [default] [JOB 20035] Generated table #246338: 931569 keys, 36621462 bytes, temperature: kWarm 2024/11/23-05:19:55.516501 140024760219200 EVENT_LOG_v1 {"time_micros": 1732310395516470, "cf_name": "default", "job": 20035, "event": "table_file_creation", "file_number": 246338, "file_size": 36621462, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3748831343, "largest_seqno": 3807171633, "table_properties": {"data_size": 30737264, "index_size": 1772160, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26083932, "raw_average_key_size": 28, "raw_value_size": 108133830, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931569, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732310390, "oldest_key_time": 0, "file_creation_time": 1732310390, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 246338, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:19:55.560360 140024760219200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246339.sst) /dev/shm/ToplingTemp/Topling-419589-tNPIxA 2024/11/23-05:19:56.554252 140024760219200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246339.sst): first pass time = 0.99's, 135.048'MB/sec 2024/11/23-05:19:57.948620 140024760219200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246339.sst): old prealloc_size = 147639500, real_size = 32622496 2024/11/23-05:19:57.949215 140024760219200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246339.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924936 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0306 GB avg-val = 117.11 avg-zval = 33.11 2024/11/23-05:19:57.950051 140024760219200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246339.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:19:58.865376 140024760219200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246339.sst): second pass time = 1.39's, 77.761'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.176'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3075.425'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.294'MB/sec write SST data time = 0.13's, 285.350'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33766.941'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 77.761'MB/sec zip pipeline throughput = 77.761'MB/sec entries = 924936 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.11 avg-zval = 33.11 usrkeys = 924936 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.11 avg-zval = 33.11 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924936 bytes = 3352912 } seq expand size = 6474552 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.4453 value = 3.5375 dict = 1.00 all = 3.6442 } Zip/UnZip{ index = 0.0957 value = 0.2827 dict = 1.00 all = 0.2744 } ---------------------------- total value len = 0.458928 GB avg = 0.124 KB (by entry num) total key len = 0.103924 GB avg = 0.028 KB total ukey len = 0.074232 GB avg = 0.020 KB total ukey num = 0.003711578 Billion total entry num = 0.003711578 Billion write speed all = 0.007641657 MB/sec (with seq num) write speed all = 0.007238530 MB/sec (without seq num) 2024/11/23-05:19:58.897911 140024760219200 [compaction_job.cc:1948] [default] [JOB 20035] Generated table #246339: 924936 keys, 36830878 bytes, temperature: kWarm 2024/11/23-05:19:58.897984 140024760219200 EVENT_LOG_v1 {"time_micros": 1732310398897941, "cf_name": "default", "job": 20035, "event": "table_file_creation", "file_number": 246339, "file_size": 36830878, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038075, "largest_seqno": 3807171644, "table_properties": {"data_size": 30620048, "index_size": 1771008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25898208, "raw_average_key_size": 28, "raw_value_size": 108319656, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924936, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732310395, "oldest_key_time": 0, "file_creation_time": 1732310395, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 246339, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:19:58.934817 140024760219200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246340.sst) /dev/shm/ToplingTemp/Topling-419589-8W3Mgz 2024/11/23-05:19:59.809289 140024760219200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246340.sst): first pass time = 0.87's, 153.491'MB/sec 2024/11/23-05:20:01.212374 140024760219200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246340.sst): old prealloc_size = 147639500, real_size = 33477072 2024/11/23-05:20:01.212976 140024760219200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246340.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933543 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0315 GB avg-val = 115.77 avg-zval = 33.70 2024/11/23-05:20:01.213789 140024760219200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246340.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:20:02.132990 140024760219200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246340.sst): second pass time = 1.40's, 77.065'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.964'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3083.685'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.295'MB/sec write SST data time = 0.13's, 290.410'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 38843.922'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.065'MB/sec zip pipeline throughput = 77.065'MB/sec entries = 933543 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.77 avg-zval = 33.70 usrkeys = 933543 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.77 avg-zval = 33.70 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933543 bytes = 3384112 } seq expand size = 6534801 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.4466 value = 3.4358 dict = 1.00 all = 3.5591 } Zip/UnZip{ index = 0.0957 value = 0.2910 dict = 1.00 all = 0.2810 } ---------------------------- total value len = 0.573541 GB avg = 0.123 KB (by entry num) total key len = 0.130063 GB avg = 0.028 KB total ukey len = 0.092902 GB avg = 0.020 KB total ukey num = 0.004645121 Billion total entry num = 0.004645121 Billion write speed all = 0.009552186 MB/sec (with seq num) write speed all = 0.009047686 MB/sec (without seq num) 2024/11/23-05:20:02.168751 140024760219200 [compaction_job.cc:1948] [default] [JOB 20035] Generated table #246340: 933543 keys, 37711624 bytes, temperature: kWarm 2024/11/23-05:20:02.168821 140024760219200 EVENT_LOG_v1 {"time_micros": 1732310402168788, "cf_name": "default", "job": 20035, "event": "table_file_creation", "file_number": 246340, "file_size": 37711624, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038106, "largest_seqno": 3807171654, "table_properties": {"data_size": 31456256, "index_size": 1787264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26139204, "raw_average_key_size": 28, "raw_value_size": 108078642, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933543, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732310398, "oldest_key_time": 0, "file_creation_time": 1732310398, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 246340, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:20:02.209784 140024760219200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246341.sst) /dev/shm/ToplingTemp/Topling-419589-IyFv9v 2024/11/23-05:20:03.124690 140024760219200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246341.sst): first pass time = 0.91's, 146.708'MB/sec 2024/11/23-05:20:04.457374 140024760219200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246341.sst): old prealloc_size = 147639500, real_size = 30725264 2024/11/23-05:20:04.457946 140024760219200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246341.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926699 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0287 GB avg-val = 116.83 avg-zval = 30.99 2024/11/23-05:20:04.458680 140024760219200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246341.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:20:05.313985 140024760219200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246341.sst): second pass time = 1.33's, 81.289'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 40.516'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3208.638'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 272.429'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35204.626'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.289'MB/sec zip pipeline throughput = 81.289'MB/sec entries = 926699 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.83 avg-zval = 30.99 usrkeys = 926699 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.83 avg-zval = 30.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926699 bytes = 3127632 } seq expand size = 6486893 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0287 GB dict = 1.08 MB all = 0.0347 GB } UnZip/Zip{ index = 10.4437 value = 3.7700 dict = 1.00 all = 3.8673 } Zip/UnZip{ index = 0.0958 value = 0.2653 dict = 1.00 all = 0.2586 } ---------------------------- total value len = 0.688298 GB avg = 0.124 KB (by entry num) total key len = 0.156011 GB avg = 0.028 KB total ukey len = 0.111436 GB avg = 0.020 KB total ukey num = 0.005571820 Billion total entry num = 0.005571820 Billion write speed all = 0.011461906 MB/sec (with seq num) write speed all = 0.010856785 MB/sec (without seq num) 2024/11/23-05:20:05.345175 140024760219200 [compaction_job.cc:1948] [default] [JOB 20035] Generated table #246341: 926699 keys, 34705774 bytes, temperature: kWarm 2024/11/23-05:20:05.345231 140024760219200 EVENT_LOG_v1 {"time_micros": 1732310405345201, "cf_name": "default", "job": 20035, "event": "table_file_creation", "file_number": 246341, "file_size": 34705774, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3674420091, "largest_seqno": 3807171669, "table_properties": {"data_size": 28718720, "index_size": 1774656, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25947572, "raw_average_key_size": 28, "raw_value_size": 108270170, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926699, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732310402, "oldest_key_time": 0, "file_creation_time": 1732310402, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 246341, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:20:05.380467 140024760219200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246342.sst) /dev/shm/ToplingTemp/Topling-419589-w6ssvw 2024/11/23-05:20:06.375920 140024760219200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246342.sst): first pass time = 1.00's, 134.836'MB/sec 2024/11/23-05:20:07.704489 140024760219200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246342.sst): old prealloc_size = 147639500, real_size = 31041904 2024/11/23-05:20:07.705042 140024760219200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246342.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926892 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0290 GB avg-val = 116.80 avg-zval = 31.34 2024/11/23-05:20:07.705870 140024760219200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246342.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:20:08.568624 140024760219200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246342.sst): second pass time = 1.33's, 81.534'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 40.801'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3306.177'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.13's, 274.197'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34523.679'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.534'MB/sec zip pipeline throughput = 81.534'MB/sec entries = 926892 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.80 avg-zval = 31.34 usrkeys = 926892 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.80 avg-zval = 31.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926892 bytes = 3360000 } seq expand size = 6488244 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0290 GB dict = 1.08 MB all = 0.0353 GB } UnZip/Zip{ index = 10.5145 value = 3.7272 dict = 1.00 all = 3.8069 } Zip/UnZip{ index = 0.0951 value = 0.2683 dict = 1.00 all = 0.2627 } ---------------------------- total value len = 0.803052 GB avg = 0.124 KB (by entry num) total key len = 0.181964 GB avg = 0.028 KB total ukey len = 0.129974 GB avg = 0.020 KB total ukey num = 0.006498712 Billion total entry num = 0.006498712 Billion write speed all = 0.013371468 MB/sec (with seq num) write speed all = 0.012665714 MB/sec (without seq num) 2024/11/23-05:20:08.600022 140024760219200 [compaction_job.cc:1948] [default] [JOB 20035] Generated table #246342: 926892 keys, 35256464 bytes, temperature: kWarm 2024/11/23-05:20:08.600087 140024760219200 EVENT_LOG_v1 {"time_micros": 1732310408600054, "cf_name": "default", "job": 20035, "event": "table_file_creation", "file_number": 246342, "file_size": 35256464, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3522904895, "largest_seqno": 3807171688, "table_properties": {"data_size": 29046944, "index_size": 1763072, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25952976, "raw_average_key_size": 28, "raw_value_size": 108264832, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926892, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732310405, "oldest_key_time": 0, "file_creation_time": 1732310405, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 246342, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:20:08.636143 140024760219200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246343.sst) /dev/shm/ToplingTemp/Topling-419589-rH69wx 2024/11/23-05:20:09.647428 140024760219200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246343.sst): first pass time = 1.01's, 132.725'MB/sec 2024/11/23-05:20:10.976415 140024760219200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246343.sst): old prealloc_size = 147639500, real_size = 31673664 2024/11/23-05:20:10.976978 140024760219200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246343.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925057 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0297 GB avg-val = 117.09 avg-zval = 32.09 2024/11/23-05:20:10.977735 140024760219200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246343.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:20:11.849219 140024760219200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246343.sst): second pass time = 1.33's, 81.575'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 41.113'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3254.161'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.310'MB/sec write SST data time = 0.13's, 284.511'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34304.167'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.575'MB/sec zip pipeline throughput = 81.575'MB/sec entries = 925057 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.09 avg-zval = 32.09 usrkeys = 925057 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.09 avg-zval = 32.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925057 bytes = 3353344 } seq expand size = 6475399 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.08 MB all = 0.0359 GB } UnZip/Zip{ index = 10.5155 value = 3.6491 dict = 1.00 all = 3.7405 } Zip/UnZip{ index = 0.0951 value = 0.2740 dict = 1.00 all = 0.2673 } ---------------------------- total value len = 0.917843 GB avg = 0.124 KB (by entry num) total key len = 0.207866 GB avg = 0.028 KB total ukey len = 0.148475 GB avg = 0.020 KB total ukey num = 0.007423769 Billion total entry num = 0.007423769 Billion write speed all = 0.015280680 MB/sec (with seq num) write speed all = 0.014474501 MB/sec (without seq num) 2024/11/23-05:20:11.883818 140024760219200 [compaction_job.cc:1948] [default] [JOB 20035] Generated table #246343: 925057 keys, 35882506 bytes, temperature: kWarm 2024/11/23-05:20:11.883883 140024760219200 EVENT_LOG_v1 {"time_micros": 1732310411883852, "cf_name": "default", "job": 20035, "event": "table_file_creation", "file_number": 246343, "file_size": 35882506, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038136, "largest_seqno": 3807171707, "table_properties": {"data_size": 29682800, "index_size": 1759424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25901596, "raw_average_key_size": 28, "raw_value_size": 108316138, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925057, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732310408, "oldest_key_time": 0, "file_creation_time": 1732310408, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 246343, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:20:11.919727 140024760219200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246344.sst) /dev/shm/ToplingTemp/Topling-419589-7SEkRw 2024/11/23-05:20:12.897597 140024760219200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246344.sst): first pass time = 0.98's, 137.261'MB/sec 2024/11/23-05:20:14.264622 140024760219200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246344.sst): old prealloc_size = 147639500, real_size = 33772000 2024/11/23-05:20:14.265182 140024760219200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246344.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933330 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0318 GB avg-val = 115.81 avg-zval = 34.02 2024/11/23-05:20:14.265978 140024760219200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246344.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:20:15.150830 140024760219200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246344.sst): second pass time = 1.37's, 79.105'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 42.486'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3334.948'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.309'MB/sec write SST data time = 0.13's, 288.212'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 29212.770'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.105'MB/sec zip pipeline throughput = 79.105'MB/sec entries = 933330 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.81 avg-zval = 34.02 usrkeys = 933330 avg-key = 20.00 avg-zkey = 1.92 avg-val = 121.56 avg-zval = 34.02 TagRS{ kinds = RS_Key0_TagN bytes = 145920 } TagArray{ size = 767661 bytes = 2878736 } seq expand size = 5373627 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.4435 value = 3.4041 dict = 1.00 all = 3.5648 } Zip/UnZip{ index = 0.0958 value = 0.2938 dict = 1.00 all = 0.2805 } ---------------------------- total value len = 1.031301 GB avg = 0.123 KB (by entry num) total key len = 0.233999 GB avg = 0.028 KB total ukey len = 0.167142 GB avg = 0.020 KB total ukey num = 0.008357099 Billion total entry num = 0.008357099 Billion write speed all = 0.017174763 MB/sec (with seq num) write speed all = 0.016267271 MB/sec (without seq num) 2024/11/23-05:20:15.184631 140024760219200 [compaction_job.cc:1948] [default] [JOB 20035] Generated table #246344: 933330 keys, 37650802 bytes, temperature: kWarm 2024/11/23-05:20:15.184696 140024760219200 EVENT_LOG_v1 {"time_micros": 1732310415184663, "cf_name": "default", "job": 20035, "event": "table_file_creation", "file_number": 246344, "file_size": 37650802, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3807171720, "table_properties": {"data_size": 31751120, "index_size": 1787392, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26133240, "raw_average_key_size": 28, "raw_value_size": 108084620, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933330, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732310411, "oldest_key_time": 0, "file_creation_time": 1732310411, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 246344, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:20:15.221947 140024760219200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246345.sst) /dev/shm/ToplingTemp/Topling-419589-pvkrPz 2024/11/23-05:20:16.146658 140024760219200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246345.sst): first pass time = 0.92's, 145.152'MB/sec 2024/11/23-05:20:17.749300 140024760219200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246345.sst): old prealloc_size = 147639500, real_size = 36468576 2024/11/23-05:20:17.749857 140024760219200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246345.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927052 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.78 avg-zval = 37.19 2024/11/23-05:20:17.749894 140024760219200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246345.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:20:18.386361 140024760219200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246345.sst): second pass time = 1.60's, 67.748'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 58.593'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3311.428'MB/sec (index lex order gen) rebuild zvType time = 0.51's, 0.451'MB/sec write SST data time = 0.12's, 305.274'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31620.997'MB/sec zip my value time = 1.60's, unzip length = 0.108'GB zip my value throughput = 67.748'MB/sec zip pipeline throughput = 67.748'MB/sec entries = 927052 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.78 avg-zval = 37.19 usrkeys = 927052 avg-key = 20.00 avg-zkey = 1.90 avg-val = 116.78 avg-zval = 37.19 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 10 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5121 value = 3.1405 dict = 1.00 all = 3.5958 } Zip/UnZip{ index = 0.0951 value = 0.3184 dict = 1.00 all = 0.2781 } ---------------------------- total value len = 1.139562 GB avg = 0.123 KB (by entry num) total key len = 0.259956 GB avg = 0.028 KB total ukey len = 0.185683 GB avg = 0.020 KB total ukey num = 0.009284151 Billion total entry num = 0.009284151 Billion write speed all = 0.018995757 MB/sec (with seq num) write speed all = 0.017987641 MB/sec (without seq num) 2024/11/23-05:20:18.421071 140024760219200 [compaction_job.cc:1948] [default] [JOB 20035] Generated table #246345: 927052 keys, 37326384 bytes, temperature: kWarm 2024/11/23-05:20:18.421135 140024760219200 EVENT_LOG_v1 {"time_micros": 1732310418421101, "cf_name": "default", "job": 20035, "event": "table_file_creation", "file_number": 246345, "file_size": 37326384, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34472848, "index_size": 1763776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25957456, "raw_average_key_size": 28, "raw_value_size": 108260352, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927052, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732310415, "oldest_key_time": 0, "file_creation_time": 1732310415, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 246345, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:20:18.456747 140024760219200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246346.sst) /dev/shm/ToplingTemp/Topling-419589-4gKjPz 2024/11/23-05:20:19.428566 140024760219200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246346.sst): first pass time = 0.97's, 138.115'MB/sec 2024/11/23-05:20:20.952114 140024760219200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246346.sst): old prealloc_size = 147639500, real_size = 36905712 2024/11/23-05:20:20.952690 140024760219200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246346.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925057 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0349 GB avg-val = 117.09 avg-zval = 37.74 2024/11/23-05:20:20.953110 140024760219200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246346.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:20:21.714089 140024760219200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246346.sst): second pass time = 1.52's, 71.147'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.76's, 51.600'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3163.726'MB/sec (index lex order gen) rebuild zvType time = 0.64's, 0.363'MB/sec write SST data time = 0.12's, 314.958'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35395.985'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 71.147'MB/sec zip pipeline throughput = 71.147'MB/sec entries = 925057 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.09 avg-zval = 37.74 usrkeys = 925057 avg-key = 20.00 avg-zkey = 1.90 avg-val = 120.03 avg-zval = 37.74 TagRS{ kinds = RS_Key0_TagN bytes = 144640 } TagArray{ size = 388287 bytes = 1407552 } seq expand size = 2718009 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0349 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.5120 value = 3.1023 dict = 1.00 all = 3.4137 } Zip/UnZip{ index = 0.0951 value = 0.3223 dict = 1.00 all = 0.2929 } ---------------------------- total value len = 1.250596 GB avg = 0.122 KB (by entry num) total key len = 0.285858 GB avg = 0.028 KB total ukey len = 0.204184 GB avg = 0.020 KB total ukey num = 0.010209208 Billion total entry num = 0.010209208 Billion write speed all = 0.020853454 MB/sec (with seq num) write speed all = 0.019744942 MB/sec (without seq num) 2024/11/23-05:20:21.748982 140024760219200 [compaction_job.cc:1948] [default] [JOB 20035] Generated table #246346: 925057 keys, 39317197 bytes, temperature: kWarm 2024/11/23-05:20:21.749040 140024760219200 EVENT_LOG_v1 {"time_micros": 1732310421749008, "cf_name": "default", "job": 20035, "event": "table_file_creation", "file_number": 246346, "file_size": 39317197, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3807171755, "table_properties": {"data_size": 34914272, "index_size": 1760000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25901596, "raw_average_key_size": 28, "raw_value_size": 108316178, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925057, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732310418, "oldest_key_time": 0, "file_creation_time": 1732310418, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 246346, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:20:21.786524 140024760219200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246347.sst) /dev/shm/ToplingTemp/Topling-419589-XtVZKy 2024/11/23-05:20:22.674360 140024760219200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246347.sst): first pass time = 0.89's, 151.180'MB/sec 2024/11/23-05:20:24.022807 140024760219200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246347.sst): old prealloc_size = 147639500, real_size = 33686544 2024/11/23-05:20:24.023317 140024760219200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246347.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924937 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0317 GB avg-val = 117.11 avg-zval = 34.26 2024/11/23-05:20:24.024119 140024760219200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246347.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:20:25.163268 140024760219200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246347.sst): second pass time = 1.35's, 80.398'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.14's, 33.127'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3622.093'MB/sec (index lex order gen) rebuild zvType time = 0.91's, 0.255'MB/sec write SST data time = 0.23's, 162.722'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31589.796'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.398'MB/sec zip pipeline throughput = 80.398'MB/sec entries = 924937 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.11 avg-zval = 34.26 usrkeys = 924937 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.11 avg-zval = 34.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924937 bytes = 3237296 } seq expand size = 6474559 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.4442 value = 3.4188 dict = 1.00 all = 3.5527 } Zip/UnZip{ index = 0.0957 value = 0.2925 dict = 1.00 all = 0.2815 } ---------------------------- total value len = 1.365390 GB avg = 0.123 KB (by entry num) total key len = 0.311756 GB avg = 0.028 KB total ukey len = 0.222683 GB avg = 0.020 KB total ukey num = 0.011134145 Billion total entry num = 0.011134145 Billion write speed all = 0.022761930 MB/sec (with seq num) write speed all = 0.021553045 MB/sec (without seq num) 2024/11/23-05:20:25.214626 140024760219200 [compaction_job.cc:1948] [default] [JOB 20035] Generated table #246347: 924937 keys, 37779450 bytes, temperature: kWarm 2024/11/23-05:20:25.214711 140024760219200 EVENT_LOG_v1 {"time_micros": 1732310425214666, "cf_name": "default", "job": 20035, "event": "table_file_creation", "file_number": 246347, "file_size": 37779450, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3631595348, "largest_seqno": 3807171768, "table_properties": {"data_size": 31683904, "index_size": 1771200, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25898236, "raw_average_key_size": 28, "raw_value_size": 108319574, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924937, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732310421, "oldest_key_time": 0, "file_creation_time": 1732310421, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 246347, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:20:25.285941 140024760219200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246348.sst) /dev/shm/ToplingTemp/Topling-419589-QOmqNz 2024/11/23-05:20:26.295461 140024760219200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246348.sst): first pass time = 1.01's, 132.959'MB/sec 2024/11/23-05:20:28.293955 140024760219200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246348.sst): old prealloc_size = 147639500, real_size = 32703328 2024/11/23-05:20:28.294468 140024760219200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246348.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928025 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0307 GB avg-val = 116.63 avg-zval = 33.09 2024/11/23-05:20:28.295234 140024760219200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246348.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:20:29.678803 140024760219200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246348.sst): second pass time = 2.00's, 54.175'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.38's, 26.494'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3591.758'MB/sec (index lex order gen) rebuild zvType time = 1.11's, 0.208'MB/sec write SST data time = 0.27's, 136.093'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 26382.349'MB/sec zip my value time = 2.00's, unzip length = 0.108'GB zip my value throughput = 54.175'MB/sec zip pipeline throughput = 127.462'MB/sec entries = 928025 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.63 avg-zval = 33.09 usrkeys = 928025 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.63 avg-zval = 33.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928025 bytes = 3132096 } seq expand size = 6496175 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5125 value = 3.5249 dict = 1.00 all = 3.6583 } Zip/UnZip{ index = 0.0951 value = 0.2837 dict = 1.00 all = 0.2734 } ---------------------------- total value len = 1.480119 GB avg = 0.123 KB (by entry num) total key len = 0.337741 GB avg = 0.028 KB total ukey len = 0.241243 GB avg = 0.020 KB total ukey num = 0.012062170 Billion total entry num = 0.012062170 Billion write speed all = 0.024670163 MB/sec (with seq num) write speed all = 0.023360598 MB/sec (without seq num) 2024/11/23-05:20:29.712704 140024760219200 [compaction_job.cc:1948] [default] [JOB 20035] Generated table #246348: 928025 keys, 36688956 bytes, temperature: kWarm 2024/11/23-05:20:29.712774 140024760219200 EVENT_LOG_v1 {"time_micros": 1732310429712733, "cf_name": "default", "job": 20035, "event": "table_file_creation", "file_number": 246348, "file_size": 36688956, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3733723468, "largest_seqno": 3807171785, "table_properties": {"data_size": 30705552, "index_size": 1765568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25984700, "raw_average_key_size": 28, "raw_value_size": 108233042, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928025, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732310425, "oldest_key_time": 0, "file_creation_time": 1732310425, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 246348, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:20:29.762796 140024760219200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246349.sst) /dev/shm/ToplingTemp/Topling-419589-l1niQy 2024/11/23-05:20:30.675055 140024760219200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246349.sst): first pass time = 0.91's, 147.132'MB/sec 2024/11/23-05:20:32.262937 140024760219200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246349.sst): old prealloc_size = 147639500, real_size = 32720144 2024/11/23-05:20:32.263629 140024760219200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246349.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932482 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0307 GB avg-val = 115.94 avg-zval = 32.92 2024/11/23-05:20:32.266559 140024760219200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246349.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:20:33.429194 140024760219200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246349.sst): second pass time = 1.57's, 68.737'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.17's, 31.584'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2663.695'MB/sec (index lex order gen) rebuild zvType time = 1.01's, 0.232'MB/sec write SST data time = 0.16's, 231.574'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 27642.663'MB/sec zip my value time = 1.57's, unzip length = 0.108'GB zip my value throughput = 68.737'MB/sec zip pipeline throughput = 68.737'MB/sec entries = 932482 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.94 avg-zval = 32.92 usrkeys = 932482 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.94 avg-zval = 32.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932482 bytes = 3263696 } seq expand size = 6527374 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.4441 value = 3.5213 dict = 1.00 all = 3.6438 } Zip/UnZip{ index = 0.0957 value = 0.2840 dict = 1.00 all = 0.2744 } ---------------------------- total value len = 1.594755 GB avg = 0.123 KB (by entry num) total key len = 0.363850 GB avg = 0.028 KB total ukey len = 0.259893 GB avg = 0.020 KB total ukey num = 0.012994652 Billion total entry num = 0.012994652 Billion write speed all = 0.026578862 MB/sec (with seq num) write speed all = 0.025168131 MB/sec (without seq num) 2024/11/23-05:20:33.463112 140024760219200 [compaction_job.cc:1948] [default] [JOB 20035] Generated table #246349: 932482 keys, 36834458 bytes, temperature: kWarm 2024/11/23-05:20:33.463174 140024760219200 EVENT_LOG_v1 {"time_micros": 1732310433463144, "cf_name": "default", "job": 20035, "event": "table_file_creation", "file_number": 246349, "file_size": 36834458, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 3807171818, "table_properties": {"data_size": 30701184, "index_size": 1785664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26109496, "raw_average_key_size": 28, "raw_value_size": 108108348, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932482, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732310429, "oldest_key_time": 0, "file_creation_time": 1732310429, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 246349, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:20:33.501886 140024760219200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246350.sst) /dev/shm/ToplingTemp/Topling-419589-sbmIux 2024/11/23-05:20:34.510702 140024760219200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246350.sst): first pass time = 1.01's, 133.050'MB/sec 2024/11/23-05:20:36.333921 140024760219200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246350.sst): old prealloc_size = 147639500, real_size = 33812128 2024/11/23-05:20:36.334595 140024760219200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246350.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931803 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0318 GB avg-val = 116.04 avg-zval = 34.13 2024/11/23-05:20:36.336198 140024760219200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246350.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:20:37.444242 140024760219200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246350.sst): second pass time = 1.82's, 59.327'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.11's, 34.261'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2870.290'MB/sec (index lex order gen) rebuild zvType time = 0.90's, 0.259'MB/sec write SST data time = 0.21's, 179.597'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 29228.904'MB/sec zip my value time = 1.82's, unzip length = 0.108'GB zip my value throughput = 59.327'MB/sec zip pipeline throughput = 131.549'MB/sec entries = 931803 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.04 avg-zval = 34.13 usrkeys = 931803 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.04 avg-zval = 34.13 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931803 bytes = 3377808 } seq expand size = 6522621 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.5134 value = 3.3995 dict = 1.00 all = 3.5283 } Zip/UnZip{ index = 0.0951 value = 0.2942 dict = 1.00 all = 0.2834 } ---------------------------- total value len = 1.709405 GB avg = 0.123 KB (by entry num) total key len = 0.389941 GB avg = 0.028 KB total ukey len = 0.278529 GB avg = 0.020 KB total ukey num = 0.013926455 Billion total entry num = 0.013926455 Billion write speed all = 0.028487199 MB/sec (with seq num) write speed all = 0.026975392 MB/sec (without seq num) 2024/11/23-05:20:37.477856 140024760219200 [compaction_job.cc:1948] [default] [JOB 20035] Generated table #246350: 931803 keys, 38040414 bytes, temperature: kWarm 2024/11/23-05:20:37.477922 140024760219200 EVENT_LOG_v1 {"time_micros": 1732310437477889, "cf_name": "default", "job": 20035, "event": "table_file_creation", "file_number": 246350, "file_size": 38040414, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3806398219, "table_properties": {"data_size": 31806416, "index_size": 1772608, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26090484, "raw_average_key_size": 28, "raw_value_size": 108127258, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931803, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732310433, "oldest_key_time": 0, "file_creation_time": 1732310433, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 246350, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:20:37.516197 140024760219200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246351.sst) /dev/shm/ToplingTemp/Topling-419589-Y5Oiax 2024/11/23-05:20:37.607000 140024760219200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246351.sst): first pass time = 0.09's, 241.370'MB/sec 2024/11/23-05:20:37.814373 140024760219200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246351.sst): old prealloc_size = 147639500, real_size = 6429808 2024/11/23-05:20:37.814517 140024760219200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246351.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 150061 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0177 GB zip-val = 0.0061 GB avg-val = 118.00 avg-zval = 40.67 2024/11/23-05:20:37.814663 140024760219200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246351.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.022 GB 2024/11/23-05:20:37.957297 140024760219200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00/246351.sst): second pass time = 0.21's, 85.594'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.14's, 49.324'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2373.840'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.308'MB/sec write SST data time = 0.02's, 333.178'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.186'MB, throughput = 6762.227'MB/sec zip my value time = 0.21's, unzip length = 0.018'GB zip my value throughput = 85.594'MB/sec zip pipeline throughput = 85.594'MB/sec entries = 150061 avg-key = 28.00 avg-zkey = 1.92 avg-val = 118.00 avg-zval = 40.67 usrkeys = 150061 avg-key = 20.00 avg-zkey = 1.92 avg-val = 125.00 avg-zval = 40.67 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 150061 bytes = 468960 } seq expand size = 1050427 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0030 GB value = 0.0177 GB dict = 0.19 MB all = 0.0219 GB } __ZipSize{ index = 0.0003 GB value = 0.0061 GB dict = 0.19 MB all = 0.0070 GB } UnZip/Zip{ index = 10.3955 value = 2.9012 dict = 1.00 all = 3.1082 } Zip/UnZip{ index = 0.0962 value = 0.3447 dict = 1.00 all = 0.3217 } ---------------------------- total value len = 1.728162 GB avg = 0.123 KB (by entry num) total key len = 0.394142 GB avg = 0.028 KB total ukey len = 0.281530 GB avg = 0.020 KB total ukey num = 0.014076516 Billion total entry num = 0.014076516 Billion write speed all = 0.028798547 MB/sec (with seq num) write speed all = 0.027270460 MB/sec (without seq num) 2024/11/23-05:20:37.970487 140024760219200 [compaction_job.cc:1948] [default] [JOB 20035] Generated table #246351: 150061 keys, 7048719 bytes, temperature: kWarm 2024/11/23-05:20:37.970542 140024760219200 EVENT_LOG_v1 {"time_micros": 1732310437970513, "cf_name": "default", "job": 20035, "event": "table_file_creation", "file_number": 246351, "file_size": 7048719, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3754573715, "largest_seqno": 3784783750, "table_properties": {"data_size": 6103392, "index_size": 288704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4201708, "raw_average_key_size": 28, "raw_value_size": 17707198, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 150061, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732310437, "oldest_key_time": 0, "file_creation_time": 1732310437, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 246351, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:20:37.980775 140024760219200 [compaction_job.cc:703] job-20035: subcompact[0], size: 0.562099 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20035/att-00 [246336,246337,246338,246339,246340,246341,246342,246343,246344,246345,246346,246347,246348,246349,246350,246351] 2024/11/23-05:20:38.107772 140024760219200 (Original Log Time 2024/11/23-05:19:40.658324) EVENT_LOG_v1 {"time_micros": 1732310380658280, "job": 20035, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [246328], "files_L3": [246269, 246270, 246271, 246272, 246273, 246274, 246275, 246276, 246277, 246278, 246279, 246280, 246281, 246282, 246290], "score": 0.999961, "input_data_size": 581930269, "oldest_snapshot_seqno": 3807189830} 2024/11/23-05:20:38.107776 140024760219200 (Original Log Time 2024/11/23-05:20:38.104102) [compaction_job.cc:2022] [default] [JOB 20035] Compacted 1@2 + 15@3 files to L3 => 562099408 bytes 2024/11/23-05:20:38.107786 140024760219200 (Original Log Time 2024/11/23-05:20:38.107705) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 16 0 0 0] max score 0.02, MB/sec: 10.2 rd, 9.8 wr, level 3, files in(1, 15) out(16 +0 blob) MB in(27.1, 527.8 +0.0 blob) out(536.1 +0.0 blob), read-write-amplify(40.2) write-amplify(19.7) OK, records in: 14158588, records dropped: 82072 output_compression: Snappy 2024/11/23-05:20:38.107791 140024760219200 (Original Log Time 2024/11/23-05:20:38.107752) EVENT_LOG_v1 {"time_micros": 1732310438107729, "cf": "default", "job": 20035, "event": "compaction_finished", "compaction_time_micros": 57322463, "compaction_time_cpu_micros": 24812400, "output_level": 3, "num_output_files": 16, "total_output_size": 562099408, "num_input_records": 14158588, "num_output_records": 14076516, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 416639786, "file_range_sync_nanos": 0, "file_fsync_nanos": 14822, "file_prepare_write_nanos": 36910, "lsm_state": [0, 0, 0, 16, 0, 0, 0]} 2024/11/23-05:20:38.228467 140024760219200 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732310380,"executesMs":57576,"compactionJobId":20035,"attempt":0,"compactionInputRawBytes":2046829940,"compactionInputZipBytes":581930269,"compactionOutputRawBytes":0,"compactionOutputZipBytes":562099408,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-c586ab6db54ae175436ce8f6b8cd360e-914905e687234932-00"}}, response = 2024/11/23-05:20:38.228539 140024760219200 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-20035/att-00: olev 3, work 57.577 s, result 6.251 ms, install 3.707 ms, input{raw 1.906 GiB zip 554.972 MiB}