2024/11/23-06:00:00.252747 140024768611904 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00: fork to child time = 0.009138 sec 2024/11/23-06:00:00.253614 140024768611904 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00 2024/11/23-06:00:00.261157 140024768611904 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00 2024/11/23-06:00:00.265735 140024768611904 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-06:00:00.266033 140024768611904 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-06:00:00.313535 140024768611904 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-20649/att-00/MANIFEST-206347 2024/11/23-06:00:00.330356 140024768611904 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-20649/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 247671, last_sequence is 3824162291, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 247473 2024/11/23-06:00:00.330362 140024768611904 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-06:00:00.330702 140024768611904 [version_set.cc:5721] Creating manifest 247671 2024/11/23-06:00:00.365680 140024768611904 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-06:00:00.384967 140024768611904 [compaction_job.cc:2437] [default] [JOB 20649] Compacting 1@2 + 16@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-06:00:00.384986 140024768611904 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 15438 Base level 2, inputs: [247625(27MB)], [247574(34MB) 247575(35MB) 247576(34MB) 247577(35MB) 247578(34MB) 247579(35MB) 247580(36MB) 247581(33MB) 247582(35MB) 247583(35MB) 247584(36MB) 247585(35MB) 247586(33MB) 247587(34MB) 247588(6109KB) 247656(35MB)] 2024/11/23-06:00:00.391112 140024768611904 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-06:00:00.391143 140024768611904 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-06:00:00.391161 140024768611904 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-06:00:00.391167 140024768611904 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-06:00:00.391178 140024768611904 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-06:00:00.832496 140024768611904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247672.sst) /dev/shm/ToplingTemp/Topling-428526-992X0k 2024/11/23-06:00:01.381492 140024768611904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247672.sst): first pass time = 0.55's, 244.498'MB/sec 2024/11/23-06:00:03.160558 140024768611904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247672.sst): old prealloc_size = 147639500, real_size = 32395552 2024/11/23-06:00:03.161250 140024768611904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247672.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927712 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0304 GB avg-val = 116.68 avg-zval = 32.77 2024/11/23-06:00:03.162095 140024768611904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247672.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:00:04.339867 140024768611904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247672.sst): second pass time = 1.78's, 60.874'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.18's, 30.849'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2639.168'MB/sec (index lex order gen) rebuild zvType time = 1.02's, 0.227'MB/sec write SST data time = 0.16's, 234.130'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35181.789'MB/sec zip my value time = 1.78's, unzip length = 0.108'GB zip my value throughput = 60.874'MB/sec zip pipeline throughput = 60.874'MB/sec entries = 927712 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.68 avg-zval = 32.77 usrkeys = 927712 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.68 avg-zval = 32.77 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927712 bytes = 3131040 } seq expand size = 6493984 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5112 value = 3.5608 dict = 1.00 all = 3.6892 } Zip/UnZip{ index = 0.0951 value = 0.2808 dict = 1.00 all = 0.2711 } ---------------------------- total value len = 0.114736 GB avg = 0.124 KB (by entry num) total key len = 0.025976 GB avg = 0.028 KB total ukey len = 0.018554 GB avg = 0.020 KB total ukey num = 0.000927712 Billion total entry num = 0.000927712 Billion write speed all = 0.001849980 MB/sec (with seq num) write speed all = 0.001752405 MB/sec (without seq num) 2024/11/23-06:00:04.373679 140024768611904 [compaction_job.cc:1948] [default] [JOB 20649] Generated table #247672: 927712 keys, 36380788 bytes, temperature: kWarm 2024/11/23-06:00:04.373767 140024768611904 EVENT_LOG_v1 {"time_micros": 1732312804373714, "cf_name": "default", "job": 20649, "event": "table_file_creation", "file_number": 247672, "file_size": 36380788, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3726348054, "largest_seqno": 3823843558, "table_properties": {"data_size": 30398288, "index_size": 1765184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25975936, "raw_average_key_size": 28, "raw_value_size": 108241876, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927712, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732312800, "oldest_key_time": 0, "file_creation_time": 1732312800, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247672, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:00:04.411480 140024768611904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247673.sst) /dev/shm/ToplingTemp/Topling-428526-NhY4xl 2024/11/23-06:00:05.602646 140024768611904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247673.sst): first pass time = 1.19's, 112.681'MB/sec 2024/11/23-06:00:07.099634 140024768611904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247673.sst): old prealloc_size = 147639500, real_size = 32976240 2024/11/23-06:00:07.100219 140024768611904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247673.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927884 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.65 avg-zval = 33.39 2024/11/23-06:00:07.101040 140024768611904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247673.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:00:08.033688 140024768611904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247673.sst): second pass time = 1.50's, 72.361'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 39.820'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3154.653'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.288'MB/sec write SST data time = 0.13's, 290.551'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35986.273'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 72.361'MB/sec zip pipeline throughput = 72.361'MB/sec entries = 927884 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.65 avg-zval = 33.39 usrkeys = 927884 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.65 avg-zval = 33.39 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927884 bytes = 3363600 } seq expand size = 6495188 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5136 value = 3.4939 dict = 1.00 all = 3.6086 } Zip/UnZip{ index = 0.0951 value = 0.2862 dict = 1.00 all = 0.2771 } ---------------------------- total value len = 0.229468 GB avg = 0.124 KB (by entry num) total key len = 0.051957 GB avg = 0.028 KB total ukey len = 0.037112 GB avg = 0.020 KB total ukey num = 0.001855596 Billion total entry num = 0.001855596 Billion write speed all = 0.003699796 MB/sec (with seq num) write speed all = 0.003504636 MB/sec (without seq num) 2024/11/23-06:00:08.068606 140024768611904 [compaction_job.cc:1948] [default] [JOB 20649] Generated table #247673: 927884 keys, 37193836 bytes, temperature: kWarm 2024/11/23-06:00:08.068669 140024768611904 EVENT_LOG_v1 {"time_micros": 1732312808068638, "cf_name": "default", "job": 20649, "event": "table_file_creation", "file_number": 247673, "file_size": 37193836, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038037, "largest_seqno": 3823843568, "table_properties": {"data_size": 30978976, "index_size": 1765120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25980752, "raw_average_key_size": 28, "raw_value_size": 108236980, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927884, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732312804, "oldest_key_time": 0, "file_creation_time": 1732312804, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247673, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:00:08.103871 140024768611904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247674.sst) /dev/shm/ToplingTemp/Topling-428526-57zXfl 2024/11/23-06:00:09.102173 140024768611904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247674.sst): first pass time = 1.00's, 134.452'MB/sec 2024/11/23-06:00:10.496007 140024768611904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247674.sst): old prealloc_size = 147639500, real_size = 32135056 2024/11/23-06:00:10.496609 140024768611904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247674.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927970 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0301 GB avg-val = 116.64 avg-zval = 32.46 2024/11/23-06:00:10.497381 140024768611904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247674.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:00:11.399877 140024768611904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247674.sst): second pass time = 1.39's, 77.689'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 39.963'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3061.215'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.300'MB/sec write SST data time = 0.13's, 279.478'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 37289.044'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 77.689'MB/sec zip pipeline throughput = 77.689'MB/sec entries = 927970 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.64 avg-zval = 32.46 usrkeys = 927970 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.64 avg-zval = 32.46 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927970 bytes = 3131920 } seq expand size = 6495790 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.4426 value = 3.5928 dict = 1.00 all = 3.7158 } Zip/UnZip{ index = 0.0958 value = 0.2783 dict = 1.00 all = 0.2691 } ---------------------------- total value len = 0.344198 GB avg = 0.124 KB (by entry num) total key len = 0.077940 GB avg = 0.028 KB total ukey len = 0.055671 GB avg = 0.020 KB total ukey num = 0.002783566 Billion total entry num = 0.002783566 Billion write speed all = 0.005549463 MB/sec (with seq num) write speed all = 0.005256720 MB/sec (without seq num) 2024/11/23-06:00:11.430013 140024768611904 [compaction_job.cc:1948] [default] [JOB 20649] Generated table #247674: 927970 keys, 36120712 bytes, temperature: kWarm 2024/11/23-06:00:11.430076 140024768611904 EVENT_LOG_v1 {"time_micros": 1732312811430043, "cf_name": "default", "job": 20649, "event": "table_file_creation", "file_number": 247674, "file_size": 36120712, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3702155843, "largest_seqno": 3823843581, "table_properties": {"data_size": 30125568, "index_size": 1777280, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25983160, "raw_average_key_size": 28, "raw_value_size": 108234608, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927970, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732312808, "oldest_key_time": 0, "file_creation_time": 1732312808, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247674, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:00:11.468230 140024768611904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247675.sst) /dev/shm/ToplingTemp/Topling-428526-UqtnYl 2024/11/23-06:00:12.375604 140024768611904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247675.sst): first pass time = 0.91's, 147.925'MB/sec 2024/11/23-06:00:13.790854 140024768611904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247675.sst): old prealloc_size = 147639500, real_size = 33308272 2024/11/23-06:00:13.791392 140024768611904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247675.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927951 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0313 GB avg-val = 116.64 avg-zval = 33.73 2024/11/23-06:00:13.792220 140024768611904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247675.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:00:14.742826 140024768611904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247675.sst): second pass time = 1.41's, 76.514'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.420'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3453.979'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.281'MB/sec write SST data time = 0.13's, 295.467'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35053.325'MB/sec zip my value time = 1.41's, unzip length = 0.108'GB zip my value throughput = 76.514'MB/sec zip pipeline throughput = 76.514'MB/sec entries = 927951 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.64 avg-zval = 33.73 usrkeys = 927951 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.64 avg-zval = 33.73 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927951 bytes = 3363840 } seq expand size = 6495657 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4446 value = 3.4581 dict = 1.00 all = 3.5767 } Zip/UnZip{ index = 0.0957 value = 0.2892 dict = 1.00 all = 0.2796 } ---------------------------- total value len = 0.458929 GB avg = 0.124 KB (by entry num) total key len = 0.103922 GB avg = 0.028 KB total ukey len = 0.074230 GB avg = 0.020 KB total ukey num = 0.003711517 Billion total entry num = 0.003711517 Billion write speed all = 0.007398968 MB/sec (with seq num) write speed all = 0.007008650 MB/sec (without seq num) 2024/11/23-06:00:14.775636 140024768611904 [compaction_job.cc:1948] [default] [JOB 20649] Generated table #247675: 927951 keys, 37526060 bytes, temperature: kWarm 2024/11/23-06:00:14.775696 140024768611904 EVENT_LOG_v1 {"time_micros": 1732312814775666, "cf_name": "default", "job": 20649, "event": "table_file_creation", "file_number": 247675, "file_size": 37526060, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038075, "largest_seqno": 3823843891, "table_properties": {"data_size": 31299232, "index_size": 1776896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25982628, "raw_average_key_size": 28, "raw_value_size": 108235130, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927951, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732312811, "oldest_key_time": 0, "file_creation_time": 1732312811, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247675, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:00:14.809984 140024768611904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247676.sst) /dev/shm/ToplingTemp/Topling-428526-uDUwDj 2024/11/23-06:00:15.850455 140024768611904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247676.sst): first pass time = 1.04's, 129.002'MB/sec 2024/11/23-06:00:17.209548 140024768611904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247676.sst): old prealloc_size = 147639500, real_size = 32444608 2024/11/23-06:00:17.210097 140024768611904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247676.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927932 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0304 GB avg-val = 116.64 avg-zval = 32.80 2024/11/23-06:00:17.210879 140024768611904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247676.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:00:18.112748 140024768611904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247676.sst): second pass time = 1.36's, 79.706'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 40.336'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3263.941'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.299'MB/sec write SST data time = 0.13's, 287.656'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 25270.566'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.706'MB/sec zip pipeline throughput = 79.706'MB/sec entries = 927932 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.64 avg-zval = 32.80 usrkeys = 927932 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.64 avg-zval = 32.80 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927932 bytes = 3131792 } seq expand size = 6495524 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.4463 value = 3.5562 dict = 1.00 all = 3.6842 } Zip/UnZip{ index = 0.0957 value = 0.2812 dict = 1.00 all = 0.2714 } ---------------------------- total value len = 0.573660 GB avg = 0.124 KB (by entry num) total key len = 0.129905 GB avg = 0.028 KB total ukey len = 0.092789 GB avg = 0.020 KB total ukey num = 0.004639449 Billion total entry num = 0.004639449 Billion write speed all = 0.009248305 MB/sec (with seq num) write speed all = 0.008760424 MB/sec (without seq num) 2024/11/23-06:00:18.147967 140024768611904 [compaction_job.cc:1948] [default] [JOB 20649] Generated table #247676: 927932 keys, 36430288 bytes, temperature: kWarm 2024/11/23-06:00:18.148032 140024768611904 EVENT_LOG_v1 {"time_micros": 1732312818147999, "cf_name": "default", "job": 20649, "event": "table_file_creation", "file_number": 247676, "file_size": 36430288, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3734180648, "largest_seqno": 3823844328, "table_properties": {"data_size": 30435888, "index_size": 1776576, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25982096, "raw_average_key_size": 28, "raw_value_size": 108235664, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927932, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732312814, "oldest_key_time": 0, "file_creation_time": 1732312814, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247676, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:00:18.186978 140024768611904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247677.sst) /dev/shm/ToplingTemp/Topling-428526-yrKY8l 2024/11/23-06:00:19.170446 140024768611904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247677.sst): first pass time = 0.98's, 136.480'MB/sec 2024/11/23-06:00:20.539616 140024768611904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247677.sst): old prealloc_size = 147639500, real_size = 33370112 2024/11/23-06:00:20.540205 140024768611904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247677.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927627 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0314 GB avg-val = 116.69 avg-zval = 33.81 2024/11/23-06:00:20.541031 140024768611904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247677.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:00:21.502919 140024768611904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247677.sst): second pass time = 1.37's, 79.096'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 39.019'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3129.343'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.278'MB/sec write SST data time = 0.13's, 290.634'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32416.987'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.096'MB/sec zip pipeline throughput = 79.096'MB/sec entries = 927627 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.69 avg-zval = 33.81 usrkeys = 927627 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.69 avg-zval = 33.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927627 bytes = 3362656 } seq expand size = 6493389 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.4474 value = 3.4514 dict = 1.00 all = 3.5709 } Zip/UnZip{ index = 0.0957 value = 0.2897 dict = 1.00 all = 0.2800 } ---------------------------- total value len = 0.688398 GB avg = 0.124 KB (by entry num) total key len = 0.155878 GB avg = 0.028 KB total ukey len = 0.111342 GB avg = 0.020 KB total ukey num = 0.005567076 Billion total entry num = 0.005567076 Billion write speed all = 0.011097447 MB/sec (with seq num) write speed all = 0.010512043 MB/sec (without seq num) 2024/11/23-06:00:21.538141 140024768611904 [compaction_job.cc:1948] [default] [JOB 20649] Generated table #247677: 927627 keys, 37586838 bytes, temperature: kWarm 2024/11/23-06:00:21.538205 140024768611904 EVENT_LOG_v1 {"time_micros": 1732312821538174, "cf_name": "default", "job": 20649, "event": "table_file_creation", "file_number": 247677, "file_size": 37586838, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038106, "largest_seqno": 3823843628, "table_properties": {"data_size": 31362224, "index_size": 1775808, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25973556, "raw_average_key_size": 28, "raw_value_size": 108244194, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927627, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732312818, "oldest_key_time": 0, "file_creation_time": 1732312818, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247677, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:00:21.573802 140024768611904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247678.sst) /dev/shm/ToplingTemp/Topling-428526-cyB8hj 2024/11/23-06:00:22.571217 140024768611904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247678.sst): first pass time = 1.00's, 134.571'MB/sec 2024/11/23-06:00:23.938380 140024768611904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247678.sst): old prealloc_size = 147639500, real_size = 33559728 2024/11/23-06:00:23.938990 140024768611904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247678.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929328 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0316 GB avg-val = 116.42 avg-zval = 33.96 2024/11/23-06:00:23.940065 140024768611904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247678.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:00:24.903437 140024768611904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247678.sst): second pass time = 1.37's, 79.203'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.97's, 39.150'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3017.760'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.282'MB/sec write SST data time = 0.14's, 270.300'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33966.456'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.203'MB/sec zip pipeline throughput = 79.203'MB/sec entries = 929328 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.42 avg-zval = 33.96 usrkeys = 929328 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.42 avg-zval = 33.96 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929328 bytes = 3368832 } seq expand size = 6505296 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.5128 value = 3.4284 dict = 1.00 all = 3.5525 } Zip/UnZip{ index = 0.0951 value = 0.2917 dict = 1.00 all = 0.2815 } ---------------------------- total value len = 0.803100 GB avg = 0.124 KB (by entry num) total key len = 0.181899 GB avg = 0.028 KB total ukey len = 0.129928 GB avg = 0.020 KB total ukey num = 0.006496404 Billion total entry num = 0.006496404 Billion write speed all = 0.012946580 MB/sec (with seq num) write speed all = 0.012263483 MB/sec (without seq num) 2024/11/23-06:00:24.937594 140024768611904 [compaction_job.cc:1948] [default] [JOB 20649] Generated table #247678: 929328 keys, 37781516 bytes, temperature: kWarm 2024/11/23-06:00:24.937657 140024768611904 EVENT_LOG_v1 {"time_micros": 1732312824937625, "cf_name": "default", "job": 20649, "event": "table_file_creation", "file_number": 247678, "file_size": 37781516, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038136, "largest_seqno": 3823844233, "table_properties": {"data_size": 31559200, "index_size": 1768000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26021184, "raw_average_key_size": 28, "raw_value_size": 108196616, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929328, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732312821, "oldest_key_time": 0, "file_creation_time": 1732312821, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247678, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:00:24.973515 140024768611904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247679.sst) /dev/shm/ToplingTemp/Topling-428526-6vbbik 2024/11/23-06:00:25.864589 140024768611904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247679.sst): first pass time = 0.89's, 150.631'MB/sec 2024/11/23-06:00:27.194014 140024768611904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247679.sst): old prealloc_size = 147639500, real_size = 31048512 2024/11/23-06:00:27.194529 140024768611904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247679.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923933 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0290 GB avg-val = 117.27 avg-zval = 31.44 2024/11/23-06:00:27.197119 140024768611904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247679.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:00:28.055143 140024768611904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247679.sst): second pass time = 1.33's, 81.633'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 40.939'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3554.184'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.316'MB/sec write SST data time = 0.13's, 270.365'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 32180.840'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.633'MB/sec zip pipeline throughput = 81.633'MB/sec entries = 923933 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.27 avg-zval = 31.44 usrkeys = 923933 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.27 avg-zval = 31.44 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923933 bytes = 3349280 } seq expand size = 6467531 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0290 GB dict = 1.08 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4453 value = 3.7299 dict = 1.00 all = 3.8073 } Zip/UnZip{ index = 0.0957 value = 0.2681 dict = 1.00 all = 0.2627 } ---------------------------- total value len = 0.917915 GB avg = 0.124 KB (by entry num) total key len = 0.207769 GB avg = 0.028 KB total ukey len = 0.148407 GB avg = 0.020 KB total ukey num = 0.007420337 Billion total entry num = 0.007420337 Billion write speed all = 0.014795099 MB/sec (with seq num) write speed all = 0.014014883 MB/sec (without seq num) 2024/11/23-06:00:28.091030 140024768611904 [compaction_job.cc:1948] [default] [JOB 20649] Generated table #247679: 923933 keys, 35253074 bytes, temperature: kWarm 2024/11/23-06:00:28.091102 140024768611904 EVENT_LOG_v1 {"time_micros": 1732312828091061, "cf_name": "default", "job": 20649, "event": "table_file_creation", "file_number": 247679, "file_size": 35253074, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3442641194, "largest_seqno": 3823843662, "table_properties": {"data_size": 29048240, "index_size": 1769088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25870124, "raw_average_key_size": 28, "raw_value_size": 108347618, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923933, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732312824, "oldest_key_time": 0, "file_creation_time": 1732312824, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247679, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:00:28.124922 140024768611904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247680.sst) /dev/shm/ToplingTemp/Topling-428526-AqkNHk 2024/11/23-06:00:29.051850 140024768611904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247680.sst): first pass time = 0.93's, 144.804'MB/sec 2024/11/23-06:00:30.427379 140024768611904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247680.sst): old prealloc_size = 147639500, real_size = 32948832 2024/11/23-06:00:30.427941 140024768611904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247680.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932706 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0309 GB avg-val = 115.90 avg-zval = 33.16 2024/11/23-06:00:30.428740 140024768611904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247680.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:00:31.333261 140024768611904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247680.sst): second pass time = 1.37's, 78.630'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 41.011'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3298.188'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.301'MB/sec write SST data time = 0.13's, 283.788'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 33404.204'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 78.630'MB/sec zip pipeline throughput = 78.630'MB/sec entries = 932706 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.90 avg-zval = 33.16 usrkeys = 932706 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.31 avg-zval = 33.16 TagRS{ kinds = RS_Key0_TagN bytes = 145840 } TagArray{ size = 854052 bytes = 3202704 } seq expand size = 5978364 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.4444 value = 3.4951 dict = 1.00 all = 3.6128 } Zip/UnZip{ index = 0.0957 value = 0.2861 dict = 1.00 all = 0.2768 } ---------------------------- total value len = 1.031995 GB avg = 0.124 KB (by entry num) total key len = 0.233885 GB avg = 0.028 KB total ukey len = 0.167061 GB avg = 0.020 KB total ukey num = 0.008353043 Billion total entry num = 0.008353043 Billion write speed all = 0.016637009 MB/sec (with seq num) write speed all = 0.015758761 MB/sec (without seq num) 2024/11/23-06:00:31.368204 140024768611904 [compaction_job.cc:1948] [default] [JOB 20649] Generated table #247680: 932706 keys, 37151038 bytes, temperature: kWarm 2024/11/23-06:00:31.368268 140024768611904 EVENT_LOG_v1 {"time_micros": 1732312831368237, "cf_name": "default", "job": 20649, "event": "table_file_creation", "file_number": 247680, "file_size": 37151038, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3823844106, "table_properties": {"data_size": 30929424, "index_size": 1786048, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26115768, "raw_average_key_size": 28, "raw_value_size": 108102048, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932706, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732312828, "oldest_key_time": 0, "file_creation_time": 1732312828, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247680, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:00:31.415053 140024768611904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247681.sst) /dev/shm/ToplingTemp/Topling-428526-yNc2Ki 2024/11/23-06:00:32.316192 140024768611904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247681.sst): first pass time = 0.90's, 148.949'MB/sec 2024/11/23-06:00:33.918728 140024768611904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247681.sst): old prealloc_size = 147639500, real_size = 36458800 2024/11/23-06:00:33.919338 140024768611904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247681.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927086 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.77 avg-zval = 37.19 2024/11/23-06:00:33.919384 140024768611904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247681.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:00:34.564572 140024768611904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247681.sst): second pass time = 1.60's, 67.585'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 57.781'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2964.604'MB/sec (index lex order gen) rebuild zvType time = 0.52's, 0.446'MB/sec write SST data time = 0.13's, 297.587'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33919.136'MB/sec zip my value time = 1.60's, unzip length = 0.108'GB zip my value throughput = 67.585'MB/sec zip pipeline throughput = 67.585'MB/sec entries = 927086 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.77 avg-zval = 37.19 usrkeys = 927086 avg-key = 20.00 avg-zkey = 1.89 avg-val = 116.77 avg-zval = 37.19 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 10 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5816 value = 3.1403 dict = 1.00 all = 3.5967 } Zip/UnZip{ index = 0.0945 value = 0.3184 dict = 1.00 all = 0.2780 } ---------------------------- total value len = 1.140255 GB avg = 0.123 KB (by entry num) total key len = 0.259844 GB avg = 0.028 KB total ukey len = 0.185603 GB avg = 0.020 KB total ukey num = 0.009280129 Billion total entry num = 0.009280129 Billion write speed all = 0.018400202 MB/sec (with seq num) write speed all = 0.017424521 MB/sec (without seq num) 2024/11/23-06:00:34.597997 140024768611904 [compaction_job.cc:1948] [default] [JOB 20649] Generated table #247681: 927086 keys, 37316882 bytes, temperature: kWarm 2024/11/23-06:00:34.598055 140024768611904 EVENT_LOG_v1 {"time_micros": 1732312834598026, "cf_name": "default", "job": 20649, "event": "table_file_creation", "file_number": 247681, "file_size": 37316882, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34474592, "index_size": 1752256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25958408, "raw_average_key_size": 28, "raw_value_size": 108259320, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927086, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732312831, "oldest_key_time": 0, "file_creation_time": 1732312831, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247681, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:00:34.641325 140024768611904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247682.sst) /dev/shm/ToplingTemp/Topling-428526-HtNyyj 2024/11/23-06:00:35.545013 140024768611904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247682.sst): first pass time = 0.90's, 148.528'MB/sec 2024/11/23-06:00:37.060971 140024768611904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247682.sst): old prealloc_size = 147639500, real_size = 36369648 2024/11/23-06:00:37.061485 140024768611904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247682.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926341 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0344 GB avg-val = 116.89 avg-zval = 37.11 2024/11/23-06:00:37.061869 140024768611904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247682.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:00:37.826279 140024768611904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247682.sst): second pass time = 1.52's, 71.463'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.77's, 50.317'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3602.735'MB/sec (index lex order gen) rebuild zvType time = 0.64's, 0.363'MB/sec write SST data time = 0.13's, 303.801'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 20995.219'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 71.463'MB/sec zip pipeline throughput = 71.463'MB/sec entries = 926341 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.89 avg-zval = 37.11 usrkeys = 926341 avg-key = 20.00 avg-zkey = 1.90 avg-val = 119.18 avg-zval = 37.11 TagRS{ kinds = RS_Key0_TagN bytes = 144840 } TagArray{ size = 302590 bytes = 1134720 } seq expand size = 2118130 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.5125 value = 3.1499 dict = 1.00 all = 3.4855 } Zip/UnZip{ index = 0.0951 value = 0.3175 dict = 1.00 all = 0.2869 } ---------------------------- total value len = 1.250653 GB avg = 0.123 KB (by entry num) total key len = 0.285781 GB avg = 0.028 KB total ukey len = 0.204129 GB avg = 0.020 KB total ukey num = 0.010206470 Billion total entry num = 0.010206470 Billion write speed all = 0.020191074 MB/sec (with seq num) write speed all = 0.019118046 MB/sec (without seq num) 2024/11/23-06:00:37.864363 140024768611904 [compaction_job.cc:1948] [default] [JOB 20649] Generated table #247682: 926341 keys, 38507503 bytes, temperature: kWarm 2024/11/23-06:00:37.864422 140024768611904 EVENT_LOG_v1 {"time_micros": 1732312837864389, "cf_name": "default", "job": 20649, "event": "table_file_creation", "file_number": 247682, "file_size": 38507503, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3823844640, "table_properties": {"data_size": 34375520, "index_size": 1762368, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25937548, "raw_average_key_size": 28, "raw_value_size": 108280250, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926341, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732312834, "oldest_key_time": 0, "file_creation_time": 1732312834, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247682, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:00:37.902976 140024768611904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247683.sst) /dev/shm/ToplingTemp/Topling-428526-EOv3Lj 2024/11/23-06:00:38.787247 140024768611904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247683.sst): first pass time = 0.88's, 151.789'MB/sec 2024/11/23-06:00:40.127514 140024768611904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247683.sst): old prealloc_size = 147639500, real_size = 32118576 2024/11/23-06:00:40.128112 140024768611904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247683.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929432 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0301 GB avg-val = 116.41 avg-zval = 32.39 2024/11/23-06:00:40.128940 140024768611904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247683.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:00:41.017200 140024768611904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247683.sst): second pass time = 1.34's, 80.764'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 40.847'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3097.728'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.306'MB/sec write SST data time = 0.13's, 279.370'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32377.100'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.764'MB/sec zip pipeline throughput = 80.764'MB/sec entries = 929432 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.41 avg-zval = 32.39 usrkeys = 929432 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.41 avg-zval = 32.39 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929432 bytes = 3369200 } seq expand size = 6506024 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.4429 value = 3.5938 dict = 1.00 all = 3.6934 } Zip/UnZip{ index = 0.0958 value = 0.2783 dict = 1.00 all = 0.2708 } ---------------------------- total value len = 1.365353 GB avg = 0.123 KB (by entry num) total key len = 0.311805 GB avg = 0.028 KB total ukey len = 0.222718 GB avg = 0.020 KB total ukey num = 0.011135902 Billion total entry num = 0.011135902 Billion write speed all = 0.022039475 MB/sec (with seq num) write speed all = 0.020868783 MB/sec (without seq num) 2024/11/23-06:00:41.049893 140024768611904 [compaction_job.cc:1948] [default] [JOB 20649] Generated table #247683: 929432 keys, 36340264 bytes, temperature: kWarm 2024/11/23-06:00:41.049959 140024768611904 EVENT_LOG_v1 {"time_micros": 1732312841049926, "cf_name": "default", "job": 20649, "event": "table_file_creation", "file_number": 247683, "file_size": 36340264, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3509327862, "largest_seqno": 3823844653, "table_properties": {"data_size": 30106016, "index_size": 1780032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26024096, "raw_average_key_size": 28, "raw_value_size": 108193760, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732312837, "oldest_key_time": 0, "file_creation_time": 1732312837, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247683, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:00:41.087005 140024768611904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247684.sst) /dev/shm/ToplingTemp/Topling-428526-lFRKCk 2024/11/23-06:00:41.861191 140024768611904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247684.sst): first pass time = 0.77's, 173.375'MB/sec 2024/11/23-06:00:43.170415 140024768611904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247684.sst): old prealloc_size = 147639500, real_size = 31416160 2024/11/23-06:00:43.170972 140024768611904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247684.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920424 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0294 GB avg-val = 117.82 avg-zval = 31.97 2024/11/23-06:00:43.171744 140024768611904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247684.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:00:44.047667 140024768611904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247684.sst): second pass time = 1.31's, 82.874'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 40.463'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3292.513'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.308'MB/sec write SST data time = 0.13's, 274.428'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 21768.690'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.874'MB/sec zip pipeline throughput = 82.874'MB/sec entries = 920424 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.82 avg-zval = 31.97 usrkeys = 920424 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.82 avg-zval = 31.97 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 920424 bytes = 3221504 } seq expand size = 6442968 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4438 value = 3.6857 dict = 1.00 all = 3.7812 } Zip/UnZip{ index = 0.0958 value = 0.2713 dict = 1.00 all = 0.2645 } ---------------------------- total value len = 1.480242 GB avg = 0.123 KB (by entry num) total key len = 0.337577 GB avg = 0.028 KB total ukey len = 0.241127 GB avg = 0.020 KB total ukey num = 0.012056326 Billion total entry num = 0.012056326 Billion write speed all = 0.023886942 MB/sec (with seq num) write speed all = 0.022619538 MB/sec (without seq num) 2024/11/23-06:00:44.079801 140024768611904 [compaction_job.cc:1948] [default] [JOB 20649] Generated table #247684: 920424 keys, 35495878 bytes, temperature: kWarm 2024/11/23-06:00:44.079867 140024768611904 EVENT_LOG_v1 {"time_micros": 1732312844079834, "cf_name": "default", "job": 20649, "event": "table_file_creation", "file_number": 247684, "file_size": 35495878, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3646560015, "largest_seqno": 3823844666, "table_properties": {"data_size": 29423248, "index_size": 1762624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25771872, "raw_average_key_size": 28, "raw_value_size": 108445972, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920424, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732312841, "oldest_key_time": 0, "file_creation_time": 1732312841, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247684, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:00:44.114305 140024768611904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247685.sst) /dev/shm/ToplingTemp/Topling-428526-SkcL1i 2024/11/23-06:00:44.978074 140024768611904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247685.sst): first pass time = 0.86's, 155.393'MB/sec 2024/11/23-06:00:46.302530 140024768611904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247685.sst): old prealloc_size = 147639500, real_size = 32317792 2024/11/23-06:00:46.303105 140024768611904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247685.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932898 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0303 GB avg-val = 115.87 avg-zval = 32.49 2024/11/23-06:00:46.303882 140024768611904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247685.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:00:47.256072 140024768611904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247685.sst): second pass time = 1.32's, 81.697'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 38.087'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3217.394'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.283'MB/sec write SST data time = 0.13's, 280.881'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34299.332'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.697'MB/sec zip pipeline throughput = 81.697'MB/sec entries = 932898 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.87 avg-zval = 32.49 usrkeys = 932898 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.87 avg-zval = 32.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932898 bytes = 3148544 } seq expand size = 6530286 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.5139 value = 3.5664 dict = 1.00 all = 3.6957 } Zip/UnZip{ index = 0.0951 value = 0.2804 dict = 1.00 all = 0.2706 } ---------------------------- total value len = 1.594869 GB avg = 0.123 KB (by entry num) total key len = 0.363698 GB avg = 0.028 KB total ukey len = 0.259784 GB avg = 0.020 KB total ukey num = 0.012989224 Billion total entry num = 0.012989224 Billion write speed all = 0.025735349 MB/sec (with seq num) write speed all = 0.024369933 MB/sec (without seq num) 2024/11/23-06:00:47.289981 140024768611904 [compaction_job.cc:1948] [default] [JOB 20649] Generated table #247685: 932898 keys, 36316956 bytes, temperature: kWarm 2024/11/23-06:00:47.290044 140024768611904 EVENT_LOG_v1 {"time_micros": 1732312847290012, "cf_name": "default", "job": 20649, "event": "table_file_creation", "file_number": 247685, "file_size": 36316956, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3736072251, "largest_seqno": 3823845415, "table_properties": {"data_size": 30309776, "index_size": 1774592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26121144, "raw_average_key_size": 28, "raw_value_size": 108096660, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932898, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732312844, "oldest_key_time": 0, "file_creation_time": 1732312844, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247685, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:00:47.326539 140024768611904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247686.sst) /dev/shm/ToplingTemp/Topling-428526-IUOgli 2024/11/23-06:00:48.165681 140024768611904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247686.sst): first pass time = 0.84's, 159.953'MB/sec 2024/11/23-06:00:49.486079 140024768611904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247686.sst): old prealloc_size = 147639500, real_size = 32136496 2024/11/23-06:00:49.486656 140024768611904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247686.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932042 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0301 GB avg-val = 116.00 avg-zval = 32.30 2024/11/23-06:00:49.487470 140024768611904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247686.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:00:50.395091 140024768611904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247686.sst): second pass time = 1.32's, 81.929'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 39.878'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3213.296'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.298'MB/sec write SST data time = 0.13's, 284.121'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31454.668'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.929'MB/sec zip pipeline throughput = 81.929'MB/sec entries = 932042 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.00 avg-zval = 32.30 usrkeys = 932042 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.00 avg-zval = 32.30 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932042 bytes = 3262160 } seq expand size = 6524294 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.3763 value = 3.5912 dict = 1.00 all = 3.7027 } Zip/UnZip{ index = 0.0964 value = 0.2785 dict = 1.00 all = 0.2701 } ---------------------------- total value len = 1.709514 GB avg = 0.123 KB (by entry num) total key len = 0.389795 GB avg = 0.028 KB total ukey len = 0.278425 GB avg = 0.020 KB total ukey num = 0.013921266 Billion total entry num = 0.013921266 Billion write speed all = 0.027583546 MB/sec (with seq num) write speed all = 0.026120216 MB/sec (without seq num) 2024/11/23-06:00:50.429092 140024768611904 [compaction_job.cc:1948] [default] [JOB 20649] Generated table #247686: 932042 keys, 36249110 bytes, temperature: kWarm 2024/11/23-06:00:50.429158 140024768611904 EVENT_LOG_v1 {"time_micros": 1732312850429123, "cf_name": "default", "job": 20649, "event": "table_file_creation", "file_number": 247686, "file_size": 36249110, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 3823845000, "table_properties": {"data_size": 30106848, "index_size": 1796480, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26097176, "raw_average_key_size": 28, "raw_value_size": 108120652, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 932042, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732312847, "oldest_key_time": 0, "file_creation_time": 1732312847, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247686, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:00:50.465387 140024768611904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247687.sst) /dev/shm/ToplingTemp/Topling-428526-qJpTbm 2024/11/23-06:00:50.625231 140024768611904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247687.sst): first pass time = 0.16's, 247.110'MB/sec 2024/11/23-06:00:51.016536 140024768611904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247687.sst): old prealloc_size = 147639500, real_size = 10923696 2024/11/23-06:00:51.016758 140024768611904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247687.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 270689 prefix = 4 raw-key = 0.0043 GB zip-key = 0.0005 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0319 GB zip-val = 0.0103 GB avg-val = 117.89 avg-zval = 38.19 2024/11/23-06:00:51.017029 140024768611904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247687.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0006 GB, waited 0.000 sec, Key+Value = 0.039 GB 2024/11/23-06:00:51.285391 140024768611904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00/247687.sst): second pass time = 0.39's, 81.735'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.27's, 44.870'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2644.442'MB/sec (index lex order gen) rebuild zvType time = 0.23's, 0.294'MB/sec write SST data time = 0.04's, 312.889'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.325'MB, throughput = 10637.996'MB/sec zip my value time = 0.39's, unzip length = 0.032'GB zip my value throughput = 81.735'MB/sec zip pipeline throughput = 81.735'MB/sec entries = 270689 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.89 avg-zval = 38.19 usrkeys = 270689 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.89 avg-zval = 38.19 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 270689 bytes = 879760 } seq expand size = 1894823 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0054 GB value = 0.0319 GB dict = 0.33 MB all = 0.0395 GB } __ZipSize{ index = 0.0005 GB value = 0.0103 GB dict = 0.33 MB all = 0.0121 GB } UnZip/Zip{ index = 10.4227 value = 3.0873 dict = 1.00 all = 3.2737 } Zip/UnZip{ index = 0.0959 value = 0.3239 dict = 1.00 all = 0.3055 } ---------------------------- total value len = 1.743320 GB avg = 0.123 KB (by entry num) total key len = 0.397375 GB avg = 0.028 KB total ukey len = 0.283839 GB avg = 0.020 KB total ukey num = 0.014191955 Billion total entry num = 0.014191955 Billion write speed all = 0.028126995 MB/sec (with seq num) write speed all = 0.026635228 MB/sec (without seq num) 2024/11/23-06:00:51.301172 140024768611904 [compaction_job.cc:1948] [default] [JOB 20649] Generated table #247687: 270689 keys, 12063085 bytes, temperature: kWarm 2024/11/23-06:00:51.301234 140024768611904 EVENT_LOG_v1 {"time_micros": 1732312851301203, "cf_name": "default", "job": 20649, "event": "table_file_creation", "file_number": 247687, "file_size": 12063085, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3764626893, "largest_seqno": 3823844718, "table_properties": {"data_size": 10336416, "index_size": 519424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 7579292, "raw_average_key_size": 28, "raw_value_size": 31911302, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 270689, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732312850, "oldest_key_time": 0, "file_creation_time": 1732312850, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247687, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:00:51.313720 140024768611904 [compaction_job.cc:703] job-20649: subcompact[0], size: 0.563714 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20649/att-00 [247672,247673,247674,247675,247676,247677,247678,247679,247680,247681,247682,247683,247684,247685,247686,247687] 2024/11/23-06:00:51.415773 140024768611904 (Original Log Time 2024/11/23-06:00:00.385038) EVENT_LOG_v1 {"time_micros": 1732312800384999, "job": 20649, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [247625], "files_L3": [247574, 247575, 247576, 247577, 247578, 247579, 247580, 247581, 247582, 247583, 247584, 247585, 247586, 247587, 247588, 247656], "score": 0.999247, "input_data_size": 587676120, "oldest_snapshot_seqno": 3824162269} 2024/11/23-06:00:51.415784 140024768611904 (Original Log Time 2024/11/23-06:00:51.414976) [compaction_job.cc:2022] [default] [JOB 20649] Compacted 1@2 + 16@3 files to L3 => 563713828 bytes 2024/11/23-06:00:51.415786 140024768611904 (Original Log Time 2024/11/23-06:00:51.415707) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 16 0 0 0] max score 0.02, MB/sec: 11.5 rd, 11.1 wr, level 3, files in(1, 16) out(16 +0 blob) MB in(27.1, 533.3 +0.0 blob) out(537.6 +0.0 blob), read-write-amplify(40.5) write-amplify(19.8) OK, records in: 14283650, records dropped: 91695 output_compression: Snappy 2024/11/23-06:00:51.415792 140024768611904 (Original Log Time 2024/11/23-06:00:51.415753) EVENT_LOG_v1 {"time_micros": 1732312851415732, "cf": "default", "job": 20649, "event": "compaction_finished", "compaction_time_micros": 50928689, "compaction_time_cpu_micros": 23109537, "output_level": 3, "num_output_files": 16, "total_output_size": 563713828, "num_input_records": 14283650, "num_output_records": 14191955, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 375248800, "file_range_sync_nanos": 0, "file_fsync_nanos": 13590, "file_prepare_write_nanos": 38668, "lsm_state": [0, 0, 0, 16, 0, 0, 0]} 2024/11/23-06:00:51.541693 140024768611904 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732312800,"executesMs":51155,"compactionJobId":20649,"attempt":0,"compactionInputRawBytes":2065778080,"compactionInputZipBytes":587676120,"compactionOutputRawBytes":0,"compactionOutputZipBytes":563713828,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-12d318b481f02705cbe4815b48b8afe0-6c5d9dd228d4462f-00"}}, response = 2024/11/23-06:00:51.541766 140024768611904 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-20649/att-00: olev 3, work 51.156 s, result 6.262 ms, install 0.835 ms, input{raw 1.924 GiB zip 560.452 MiB}