2024/11/23-15:28:26.294340 139957223556672 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00: fork to child time = 0.007760 sec 2024/11/23-15:28:26.295109 139957223556672 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00 2024/11/23-15:28:26.302770 139957223556672 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00 2024/11/23-15:28:26.303697 139957223556672 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-15:28:26.303993 139957223556672 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-15:28:26.359513 139957223556672 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-29323/att-00/MANIFEST-206347 2024/11/23-15:28:26.403808 139957223556672 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-29323/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 266572, last_sequence is 4068547984, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 266411 2024/11/23-15:28:26.403817 139957223556672 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-15:28:26.403989 139957223556672 [version_set.cc:5721] Creating manifest 266572 2024/11/23-15:28:26.439270 139957223556672 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-15:28:26.460935 139957223556672 [compaction_job.cc:2437] [default] [JOB 29323] Compacting 1@2 + 25@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-15:28:26.460956 139957223556672 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 21916 Base level 2, inputs: [266541(27MB)], [266567(38MB) 266568(3897KB) 266496(36MB) 266497(33MB) 266498(34MB) 266499(33MB) 266500(34MB) 266501(34MB) 266502(35MB) 266503(34MB) 266504(35MB) 266505(37MB) 266506(35MB) 266507(37MB) 266508(34MB) 266509(35MB) 266510(37MB) 266511(35MB) 266512(37MB) 266513(35MB) 266514(35MB) 266515(36MB) 266516(37MB) 266517(39MB) 266569(38MB)] 2024/11/23-15:28:26.467626 139957223556672 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-15:28:26.467656 139957223556672 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-15:28:26.467676 139957223556672 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-15:28:26.467682 139957223556672 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-15:28:26.467693 139957223556672 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-15:28:27.218794 139957223556672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266573.sst) /dev/shm/ToplingTemp/Topling-94870-QAFEBP 2024/11/23-15:28:28.357203 139957223556672 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266573.sst): first pass time = 1.14's, 117.906'MB/sec 2024/11/23-15:28:29.865913 139957223556672 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266573.sst): old prealloc_size = 147639500, real_size = 36168016 2024/11/23-15:28:29.866641 139957223556672 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266573.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924553 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0342 GB avg-val = 117.17 avg-zval = 36.98 2024/11/23-15:28:29.867531 139957223556672 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266573.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:28:31.304802 139957223556672 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266573.sst): second pass time = 1.51's, 71.890'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.44's, 28.063'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2469.826'MB/sec (index lex order gen) rebuild zvType time = 1.18's, 0.196'MB/sec write SST data time = 0.26's, 155.884'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 36491.429'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.890'MB/sec zip pipeline throughput = 71.890'MB/sec entries = 924553 avg-key = 28.00 avg-zkey = 1.89 avg-val = 117.17 avg-zval = 36.98 usrkeys = 924553 avg-key = 20.00 avg-zkey = 1.89 avg-val = 124.17 avg-zval = 36.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924553 bytes = 3351520 } seq expand size = 6471871 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0342 GB dict = 1.08 MB all = 0.0404 GB } UnZip/Zip{ index = 10.5825 value = 3.1685 dict = 1.00 all = 3.3239 } Zip/UnZip{ index = 0.0945 value = 0.3156 dict = 1.00 all = 0.3008 } ---------------------------- total value len = 8.963422 GB avg = 0.168 KB (by entry num) total key len = 1.466709 GB avg = 0.028 KB total ukey len = 1.016369 GB avg = 0.015 KB total ukey num = 0.066474988 Billion total entry num = 0.053282717 Billion write speed all = 0.441971726 MB/sec (with seq num) write speed all = 0.423909092 MB/sec (without seq num) 2024/11/23-15:28:31.356904 139957223556672 [compaction_job.cc:1948] [default] [JOB 29323] Generated table #266573: 924553 keys, 40379396 bytes, temperature: kWarm 2024/11/23-15:28:31.357042 139957223556672 EVENT_LOG_v1 {"time_micros": 1732346911356952, "cf_name": "default", "job": 29323, "event": "table_file_creation", "file_number": 266573, "file_size": 40379396, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3736072251, "largest_seqno": 4068340934, "table_properties": {"data_size": 34189376, "index_size": 1747328, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25887484, "raw_average_key_size": 28, "raw_value_size": 108330378, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924553, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346907, "oldest_key_time": 0, "file_creation_time": 1732346907, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266573, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:28:31.428461 139957223556672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266574.sst) /dev/shm/ToplingTemp/Topling-94870-Zvs3KQ 2024/11/23-15:28:32.674161 139957223556672 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266574.sst): first pass time = 1.25's, 107.750'MB/sec 2024/11/23-15:28:34.174103 139957223556672 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266574.sst): old prealloc_size = 147639500, real_size = 34693760 2024/11/23-15:28:34.174820 139957223556672 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266574.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929749 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0327 GB avg-val = 116.36 avg-zval = 35.14 2024/11/23-15:28:34.175720 139957223556672 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266574.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:28:35.084684 139957223556672 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266574.sst): second pass time = 1.50's, 72.165'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 42.867'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2583.976'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.309'MB/sec write SST data time = 0.16's, 246.338'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 27796.090'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 72.165'MB/sec zip pipeline throughput = 72.165'MB/sec entries = 929749 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.36 avg-zval = 35.14 usrkeys = 929749 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.36 avg-zval = 35.14 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929749 bytes = 3486576 } seq expand size = 6508243 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0327 GB dict = 1.08 MB all = 0.0390 GB } UnZip/Zip{ index = 10.3770 value = 3.3115 dict = 1.00 all = 3.4386 } Zip/UnZip{ index = 0.0964 value = 0.3020 dict = 1.00 all = 0.2908 } ---------------------------- total value len = 9.078115 GB avg = 0.167 KB (by entry num) total key len = 1.492742 GB avg = 0.028 KB total ukey len = 1.034964 GB avg = 0.015 KB total ukey num = 0.067404737 Billion total entry num = 0.054212466 Billion write speed all = 0.447863192 MB/sec (with seq num) write speed all = 0.429488320 MB/sec (without seq num) 2024/11/23-15:28:35.132900 139957223556672 [compaction_job.cc:1948] [default] [JOB 29323] Generated table #266574: 929749 keys, 39032816 bytes, temperature: kWarm 2024/11/23-15:28:35.132961 139957223556672 EVENT_LOG_v1 {"time_micros": 1732346915132928, "cf_name": "default", "job": 29323, "event": "table_file_creation", "file_number": 266574, "file_size": 39032816, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 4012776149, "table_properties": {"data_size": 32669232, "index_size": 1791936, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26032972, "raw_average_key_size": 28, "raw_value_size": 108184894, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929749, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346911, "oldest_key_time": 0, "file_creation_time": 1732346911, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266574, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:28:35.185242 139957223556672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266575.sst) /dev/shm/ToplingTemp/Topling-94870-SBNPyQ 2024/11/23-15:28:36.320616 139957223556672 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266575.sst): first pass time = 1.14's, 118.219'MB/sec 2024/11/23-15:28:37.790955 139957223556672 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266575.sst): old prealloc_size = 147639500, real_size = 31434240 2024/11/23-15:28:37.791680 139957223556672 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266575.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933773 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0294 GB avg-val = 115.74 avg-zval = 31.50 2024/11/23-15:28:37.792571 139957223556672 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266575.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:28:38.713040 139957223556672 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266575.sst): second pass time = 1.47's, 73.547'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 38.685'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2588.452'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.305'MB/sec write SST data time = 0.16's, 230.077'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34651.175'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.547'MB/sec zip pipeline throughput = 73.547'MB/sec entries = 933773 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.74 avg-zval = 31.50 usrkeys = 933773 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.74 avg-zval = 31.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933773 bytes = 3384944 } seq expand size = 6536411 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.4451 value = 3.6743 dict = 1.00 all = 3.7628 } Zip/UnZip{ index = 0.0957 value = 0.2722 dict = 1.00 all = 0.2658 } ---------------------------- total value len = 9.192724 GB avg = 0.167 KB (by entry num) total key len = 1.518887 GB avg = 0.028 KB total ukey len = 1.053640 GB avg = 0.015 KB total ukey num = 0.068338510 Billion total entry num = 0.055146239 Billion write speed all = 0.453756873 MB/sec (with seq num) write speed all = 0.435068379 MB/sec (without seq num) 2024/11/23-15:28:38.758267 139957223556672 [compaction_job.cc:1948] [default] [JOB 29323] Generated table #266575: 933773 keys, 35669734 bytes, temperature: kWarm 2024/11/23-15:28:38.758335 139957223556672 EVENT_LOG_v1 {"time_micros": 1732346918758301, "cf_name": "default", "job": 29323, "event": "table_file_creation", "file_number": 266575, "file_size": 35669734, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3563652023, "largest_seqno": 4012776165, "table_properties": {"data_size": 29412656, "index_size": 1787968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26145644, "raw_average_key_size": 28, "raw_value_size": 108072122, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933773, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346915, "oldest_key_time": 0, "file_creation_time": 1732346915, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266575, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:28:38.794213 139957223556672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266576.sst) /dev/shm/ToplingTemp/Topling-94870-asIxuQ 2024/11/23-15:28:39.948882 139957223556672 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266576.sst): first pass time = 1.15's, 116.243'MB/sec 2024/11/23-15:28:41.451150 139957223556672 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266576.sst): old prealloc_size = 147639500, real_size = 31897152 2024/11/23-15:28:41.451740 139957223556672 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266576.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932556 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0299 GB avg-val = 115.92 avg-zval = 32.04 2024/11/23-15:28:41.452506 139957223556672 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266576.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:28:42.292788 139957223556672 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266576.sst): second pass time = 1.50's, 71.999'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.789'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3133.987'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.326'MB/sec write SST data time = 0.13's, 286.894'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34784.767'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 71.999'MB/sec zip pipeline throughput = 71.999'MB/sec entries = 932556 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.92 avg-zval = 32.04 usrkeys = 932556 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.92 avg-zval = 32.04 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932556 bytes = 3263968 } seq expand size = 6527892 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.4457 value = 3.6182 dict = 1.00 all = 3.7271 } Zip/UnZip{ index = 0.0957 value = 0.2764 dict = 1.00 all = 0.2683 } ---------------------------- total value len = 9.307358 GB avg = 0.166 KB (by entry num) total key len = 1.544999 GB avg = 0.028 KB total ukey len = 1.072291 GB avg = 0.015 KB total ukey num = 0.069271066 Billion total entry num = 0.056078795 Billion write speed all = 0.459649332 MB/sec (with seq num) write speed all = 0.440647686 MB/sec (without seq num) 2024/11/23-15:28:42.338678 139957223556672 [compaction_job.cc:1948] [default] [JOB 29323] Generated table #266576: 932556 keys, 36011646 bytes, temperature: kWarm 2024/11/23-15:28:42.338737 139957223556672 EVENT_LOG_v1 {"time_micros": 1732346922338705, "cf_name": "default", "job": 29323, "event": "table_file_creation", "file_number": 266576, "file_size": 36011646, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3759280178, "largest_seqno": 4012776176, "table_properties": {"data_size": 29878320, "index_size": 1785536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26111568, "raw_average_key_size": 28, "raw_value_size": 108106300, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932556, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346918, "oldest_key_time": 0, "file_creation_time": 1732346918, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266576, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:28:42.378393 139957223556672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266577.sst) /dev/shm/ToplingTemp/Topling-94870-p5OaRN 2024/11/23-15:28:43.630878 139957223556672 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266577.sst): first pass time = 1.25's, 107.165'MB/sec 2024/11/23-15:28:45.117461 139957223556672 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266577.sst): old prealloc_size = 147639500, real_size = 31078144 2024/11/23-15:28:45.118194 139957223556672 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266577.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932237 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0291 GB avg-val = 115.97 avg-zval = 31.18 2024/11/23-15:28:45.119050 139957223556672 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266577.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:28:46.010312 139957223556672 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266577.sst): second pass time = 1.49's, 72.772'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 39.220'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2489.888'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.16's, 217.928'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 37915.676'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.772'MB/sec zip pipeline throughput = 72.772'MB/sec entries = 932237 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.97 avg-zval = 31.18 usrkeys = 932237 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.28 avg-zval = 31.18 TagRS{ kinds = RS_Key0_TagN bytes = 145760 } TagArray{ size = 839627 bytes = 2938704 } seq expand size = 5877389 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.5133 value = 3.7189 dict = 1.00 all = 3.8329 } Zip/UnZip{ index = 0.0951 value = 0.2689 dict = 1.00 all = 0.2609 } ---------------------------- total value len = 9.421350 GB avg = 0.165 KB (by entry num) total key len = 1.571101 GB avg = 0.028 KB total ukey len = 1.090935 GB avg = 0.016 KB total ukey num = 0.070203303 Billion total entry num = 0.057011032 Billion write speed all = 0.465509740 MB/sec (with seq num) write speed all = 0.446195256 MB/sec (without seq num) 2024/11/23-15:28:46.060764 139957223556672 [compaction_job.cc:1948] [default] [JOB 29323] Generated table #266577: 932237 keys, 35017014 bytes, temperature: kWarm 2024/11/23-15:28:46.060834 139957223556672 EVENT_LOG_v1 {"time_micros": 1732346926060798, "cf_name": "default", "job": 29323, "event": "table_file_creation", "file_number": 266577, "file_size": 35017014, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776192, "table_properties": {"data_size": 29071472, "index_size": 1773440, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26102636, "raw_average_key_size": 28, "raw_value_size": 108115190, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932237, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346922, "oldest_key_time": 0, "file_creation_time": 1732346922, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266577, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:28:46.097773 139957223556672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266578.sst) /dev/shm/ToplingTemp/Topling-94870-F1Yk1Q 2024/11/23-15:28:47.168918 139957223556672 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266578.sst): first pass time = 1.07's, 125.308'MB/sec 2024/11/23-15:28:48.742452 139957223556672 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266578.sst): old prealloc_size = 147639500, real_size = 35002128 2024/11/23-15:28:48.743309 139957223556672 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266578.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1081649 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1039 GB zip-val = 0.0327 GB avg-val = 96.09 avg-zval = 30.20 2024/11/23-15:28:48.743368 139957223556672 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266578.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:28:49.441557 139957223556672 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266578.sst): second pass time = 1.57's, 66.090'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.70's, 51.178'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2481.877'MB/sec (index lex order gen) rebuild zvType time = 0.55's, 0.496'MB/sec write SST data time = 0.15's, 234.218'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.036'MB, throughput = 25985.572'MB/sec zip my value time = 1.57's, unzip length = 0.104'GB zip my value throughput = 66.090'MB/sec zip pipeline throughput = 66.090'MB/sec entries = 1081649 avg-key = 28.00 avg-zkey = 1.91 avg-val = 96.09 avg-zval = 30.20 usrkeys = 1081649 avg-key = 20.00 avg-zkey = 1.91 avg-val = 96.09 avg-zval = 30.20 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0216 GB value = 0.1039 GB dict = 1.04 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0327 GB dict = 1.04 MB all = 0.0358 GB } UnZip/Zip{ index = 10.4468 value = 3.1822 dict = 1.00 all = 3.7514 } Zip/UnZip{ index = 0.0957 value = 0.3143 dict = 1.00 all = 0.2666 } ---------------------------- total value len = 9.525282 GB avg = 0.164 KB (by entry num) total key len = 1.601388 GB avg = 0.028 KB total ukey len = 1.112568 GB avg = 0.016 KB total ukey num = 0.071284952 Billion total entry num = 0.058092681 Billion write speed all = 0.471125155 MB/sec (with seq num) write speed all = 0.451447085 MB/sec (without seq num) 2024/11/23-15:28:49.490762 139957223556672 [compaction_job.cc:1948] [default] [JOB 29323] Generated table #266578: 1081649 keys, 35778031 bytes, temperature: kWarm 2024/11/23-15:28:49.490830 139957223556672 EVENT_LOG_v1 {"time_micros": 1732346929490795, "cf_name": "default", "job": 29323, "event": "table_file_creation", "file_number": 266578, "file_size": 35778031, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 32660736, "index_size": 2070784, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30286172, "raw_average_key_size": 28, "raw_value_size": 103931643, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1081649, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346926, "oldest_key_time": 0, "file_creation_time": 1732346926, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266578, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:28:49.527739 139957223556672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266579.sst) /dev/shm/ToplingTemp/Topling-94870-oE0VOQ 2024/11/23-15:28:50.629677 139957223556672 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266579.sst): first pass time = 1.10's, 121.806'MB/sec 2024/11/23-15:28:52.053443 139957223556672 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266579.sst): old prealloc_size = 147639500, real_size = 33614128 2024/11/23-15:28:52.054173 139957223556672 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266579.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922680 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0316 GB avg-val = 117.47 avg-zval = 34.27 2024/11/23-15:28:52.054936 139957223556672 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266579.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:28:53.081868 139957223556672 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266579.sst): second pass time = 1.42's, 76.208'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 36.248'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2492.098'MB/sec (index lex order gen) rebuild zvType time = 0.86's, 0.269'MB/sec write SST data time = 0.17's, 218.670'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 35362.541'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 76.208'MB/sec zip pipeline throughput = 76.208'MB/sec entries = 922680 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.47 avg-zval = 34.27 usrkeys = 922680 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.84 avg-zval = 34.27 TagRS{ kinds = RS_Key0_TagN bytes = 144280 } TagArray{ size = 708536 bytes = 2657024 } seq expand size = 4959752 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4455 value = 3.4280 dict = 1.00 all = 3.6005 } Zip/UnZip{ index = 0.0957 value = 0.2917 dict = 1.00 all = 0.2777 } ---------------------------- total value len = 9.638625 GB avg = 0.163 KB (by entry num) total key len = 1.627223 GB avg = 0.028 KB total ukey len = 1.131022 GB avg = 0.016 KB total ukey num = 0.072207632 Billion total entry num = 0.059015361 Billion write speed all = 0.476944691 MB/sec (with seq num) write speed all = 0.456957157 MB/sec (without seq num) 2024/11/23-15:28:53.131575 139957223556672 [compaction_job.cc:1948] [default] [JOB 29323] Generated table #266579: 922680 keys, 37277446 bytes, temperature: kWarm 2024/11/23-15:28:53.131657 139957223556672 EVENT_LOG_v1 {"time_micros": 1732346933131612, "cf_name": "default", "job": 29323, "event": "table_file_creation", "file_number": 266579, "file_size": 37277446, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012775885, "table_properties": {"data_size": 31616608, "index_size": 1766656, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25835040, "raw_average_key_size": 28, "raw_value_size": 108382740, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922680, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346929, "oldest_key_time": 0, "file_creation_time": 1732346929, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266579, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:28:53.167501 139957223556672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266580.sst) /dev/shm/ToplingTemp/Topling-94870-zGId9O 2024/11/23-15:28:54.396355 139957223556672 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266580.sst): first pass time = 1.23's, 109.226'MB/sec 2024/11/23-15:28:56.020242 139957223556672 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266580.sst): old prealloc_size = 147639500, real_size = 32904240 2024/11/23-15:28:56.020819 139957223556672 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266580.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930646 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.22 avg-zval = 33.20 2024/11/23-15:28:56.021598 139957223556672 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266580.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:28:56.905182 139957223556672 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266580.sst): second pass time = 1.62's, 66.635'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 41.958'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3201.615'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.308'MB/sec write SST data time = 0.13's, 288.518'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 28128.425'MB/sec zip my value time = 1.62's, unzip length = 0.108'GB zip my value throughput = 66.635'MB/sec zip pipeline throughput = 77.103'MB/sec entries = 930646 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.22 avg-zval = 33.20 usrkeys = 930646 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.22 avg-zval = 33.20 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930646 bytes = 3373600 } seq expand size = 6514522 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5136 value = 3.5002 dict = 1.00 all = 3.6149 } Zip/UnZip{ index = 0.0951 value = 0.2857 dict = 1.00 all = 0.2766 } ---------------------------- total value len = 9.753299 GB avg = 0.163 KB (by entry num) total key len = 1.653281 GB avg = 0.028 KB total ukey len = 1.149635 GB avg = 0.016 KB total ukey num = 0.073138278 Billion total entry num = 0.059946007 Billion write speed all = 0.482824504 MB/sec (with seq num) write speed all = 0.462525061 MB/sec (without seq num) 2024/11/23-15:28:56.954500 139957223556672 [compaction_job.cc:1948] [default] [JOB 29323] Generated table #266580: 930646 keys, 37129074 bytes, temperature: kWarm 2024/11/23-15:28:56.954590 139957223556672 EVENT_LOG_v1 {"time_micros": 1732346936954542, "cf_name": "default", "job": 29323, "event": "table_file_creation", "file_number": 266580, "file_size": 37129074, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3487497056, "largest_seqno": 4012775902, "table_properties": {"data_size": 30901024, "index_size": 1770368, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26058088, "raw_average_key_size": 28, "raw_value_size": 108159664, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930646, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346933, "oldest_key_time": 0, "file_creation_time": 1732346933, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266580, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:28:57.026153 139957223556672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266581.sst) /dev/shm/ToplingTemp/Topling-94870-2Nl17M 2024/11/23-15:28:58.096590 139957223556672 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266581.sst): first pass time = 1.07's, 125.393'MB/sec 2024/11/23-15:28:59.442527 139957223556672 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266581.sst): old prealloc_size = 147639500, real_size = 32809360 2024/11/23-15:28:59.443113 139957223556672 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266581.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932240 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 115.97 avg-zval = 33.04 2024/11/23-15:28:59.444863 139957223556672 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266581.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:29:00.308162 139957223556672 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266581.sst): second pass time = 1.35's, 80.375'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.655'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3107.958'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 282.930'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35193.508'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.375'MB/sec zip pipeline throughput = 80.375'MB/sec entries = 932240 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.97 avg-zval = 33.04 usrkeys = 932240 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.97 avg-zval = 33.04 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932240 bytes = 3262848 } seq expand size = 6525680 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5118 value = 3.5100 dict = 1.00 all = 3.6351 } Zip/UnZip{ index = 0.0951 value = 0.2849 dict = 1.00 all = 0.2751 } ---------------------------- total value len = 9.867940 GB avg = 0.162 KB (by entry num) total key len = 1.679383 GB avg = 0.028 KB total ukey len = 1.168280 GB avg = 0.016 KB total ukey num = 0.074070518 Billion total entry num = 0.060878247 Billion write speed all = 0.488711581 MB/sec (with seq num) write speed all = 0.468099424 MB/sec (without seq num) 2024/11/23-15:29:00.355478 139957223556672 [compaction_job.cc:1948] [default] [JOB 29323] Generated table #266581: 932240 keys, 36922774 bytes, temperature: kWarm 2024/11/23-15:29:00.355551 139957223556672 EVENT_LOG_v1 {"time_micros": 1732346940355514, "cf_name": "default", "job": 29323, "event": "table_file_creation", "file_number": 266581, "file_size": 36922774, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3807172320, "largest_seqno": 4012775918, "table_properties": {"data_size": 30802432, "index_size": 1773696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26102720, "raw_average_key_size": 28, "raw_value_size": 108115060, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932240, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346937, "oldest_key_time": 0, "file_creation_time": 1732346937, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266581, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:29:00.393147 139957223556672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266582.sst) /dev/shm/ToplingTemp/Topling-94870-ShVqXP 2024/11/23-15:29:01.391044 139957223556672 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266582.sst): first pass time = 1.00's, 134.508'MB/sec 2024/11/23-15:29:02.715503 139957223556672 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266582.sst): old prealloc_size = 147639500, real_size = 33126464 2024/11/23-15:29:02.716128 139957223556672 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266582.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929673 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.37 avg-zval = 33.48 2024/11/23-15:29:02.717920 139957223556672 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266582.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:29:03.573701 139957223556672 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266582.sst): second pass time = 1.32's, 81.731'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.657'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2952.582'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 291.907'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 31702.894'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.731'MB/sec zip pipeline throughput = 81.731'MB/sec entries = 929673 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.37 avg-zval = 33.48 usrkeys = 929673 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.37 avg-zval = 33.48 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929673 bytes = 3486288 } seq expand size = 6507711 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5132 value = 3.4759 dict = 1.00 all = 3.5825 } Zip/UnZip{ index = 0.0951 value = 0.2877 dict = 1.00 all = 0.2791 } ---------------------------- total value len = 9.982634 GB avg = 0.162 KB (by entry num) total key len = 1.705414 GB avg = 0.028 KB total ukey len = 1.186873 GB avg = 0.016 KB total ukey num = 0.075000191 Billion total entry num = 0.061807920 Billion write speed all = 0.494599080 MB/sec (with seq num) write speed all = 0.473675046 MB/sec (without seq num) 2024/11/23-15:29:03.619913 139957223556672 [compaction_job.cc:1948] [default] [JOB 29323] Generated table #266582: 929673 keys, 37465200 bytes, temperature: kWarm 2024/11/23-15:29:03.620000 139957223556672 EVENT_LOG_v1 {"time_micros": 1732346943619949, "cf_name": "default", "job": 29323, "event": "table_file_creation", "file_number": 266582, "file_size": 37465200, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 4012774578, "table_properties": {"data_size": 31125296, "index_size": 1768576, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26030844, "raw_average_key_size": 28, "raw_value_size": 108186946, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929673, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346940, "oldest_key_time": 0, "file_creation_time": 1732346940, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266582, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:29:03.659952 139957223556672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266583.sst) /dev/shm/ToplingTemp/Topling-94870-U53YGQ 2024/11/23-15:29:04.594004 139957223556672 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266583.sst): first pass time = 0.93's, 143.705'MB/sec 2024/11/23-15:29:05.903207 139957223556672 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266583.sst): old prealloc_size = 147639500, real_size = 34725280 2024/11/23-15:29:05.903790 139957223556672 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266583.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928829 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0327 GB avg-val = 116.50 avg-zval = 35.22 2024/11/23-15:29:05.905991 139957223556672 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266583.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:29:06.753457 139957223556672 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266583.sst): second pass time = 1.31's, 82.702'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.806'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3190.030'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 295.812'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33426.793'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.702'MB/sec zip pipeline throughput = 82.702'MB/sec entries = 928829 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.50 avg-zval = 35.22 usrkeys = 928829 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.50 avg-zval = 35.22 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928829 bytes = 3367024 } seq expand size = 6501803 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0327 GB dict = 1.08 MB all = 0.0389 GB } UnZip/Zip{ index = 10.4451 value = 3.3077 dict = 1.00 all = 3.4463 } Zip/UnZip{ index = 0.0957 value = 0.3023 dict = 1.00 all = 0.2902 } ---------------------------- total value len = 10.097347 GB avg = 0.161 KB (by entry num) total key len = 1.731421 GB avg = 0.028 KB total ukey len = 1.205450 GB avg = 0.016 KB total ukey num = 0.075929020 Billion total entry num = 0.062736749 Billion write speed all = 0.500486520 MB/sec (with seq num) write speed all = 0.479250904 MB/sec (without seq num) 2024/11/23-15:29:06.802517 139957223556672 [compaction_job.cc:1948] [default] [JOB 29323] Generated table #266583: 928829 keys, 38945524 bytes, temperature: kWarm 2024/11/23-15:29:06.802589 139957223556672 EVENT_LOG_v1 {"time_micros": 1732346946802551, "cf_name": "default", "job": 29323, "event": "table_file_creation", "file_number": 266583, "file_size": 38945524, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775967, "table_properties": {"data_size": 32714384, "index_size": 1778496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26007212, "raw_average_key_size": 28, "raw_value_size": 108210614, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928829, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346943, "oldest_key_time": 0, "file_creation_time": 1732346943, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266583, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:29:06.863233 139957223556672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266584.sst) /dev/shm/ToplingTemp/Topling-94870-JiFhsN 2024/11/23-15:29:07.692866 139957223556672 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266584.sst): first pass time = 0.83's, 161.789'MB/sec 2024/11/23-15:29:09.005498 139957223556672 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266584.sst): old prealloc_size = 147639500, real_size = 33681312 2024/11/23-15:29:09.006120 139957223556672 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266584.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926885 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0317 GB avg-val = 116.81 avg-zval = 34.19 2024/11/23-15:29:09.007598 139957223556672 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266584.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:29:09.848907 139957223556672 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266584.sst): second pass time = 1.31's, 82.527'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.933'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2985.285'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.324'MB/sec write SST data time = 0.13's, 294.974'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 29868.348'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.527'MB/sec zip pipeline throughput = 82.527'MB/sec entries = 926885 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.81 avg-zval = 34.19 usrkeys = 926885 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.81 avg-zval = 34.19 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926885 bytes = 3359968 } seq expand size = 6488195 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.5148 value = 3.4168 dict = 1.00 all = 3.5418 } Zip/UnZip{ index = 0.0951 value = 0.2927 dict = 1.00 all = 0.2823 } ---------------------------- total value len = 10.212100 GB avg = 0.160 KB (by entry num) total key len = 1.757374 GB avg = 0.028 KB total ukey len = 1.223987 GB avg = 0.016 KB total ukey num = 0.076855905 Billion total entry num = 0.063663634 Billion write speed all = 0.506373606 MB/sec (with seq num) write speed all = 0.484827072 MB/sec (without seq num) 2024/11/23-15:29:09.895500 139957223556672 [compaction_job.cc:1948] [default] [JOB 29323] Generated table #266584: 926885 keys, 37895744 bytes, temperature: kWarm 2024/11/23-15:29:09.895570 139957223556672 EVENT_LOG_v1 {"time_micros": 1732346949895534, "cf_name": "default", "job": 29323, "event": "table_file_creation", "file_number": 266584, "file_size": 37895744, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 4012776021, "table_properties": {"data_size": 31686416, "index_size": 1763008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25952780, "raw_average_key_size": 28, "raw_value_size": 108265030, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926885, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346946, "oldest_key_time": 0, "file_creation_time": 1732346946, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266584, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:29:09.931933 139957223556672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266585.sst) /dev/shm/ToplingTemp/Topling-94870-ClOocN 2024/11/23-15:29:10.976147 139957223556672 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266585.sst): first pass time = 1.04's, 128.540'MB/sec 2024/11/23-15:29:12.307699 139957223556672 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266585.sst): old prealloc_size = 147639500, real_size = 36105408 2024/11/23-15:29:12.308267 139957223556672 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266585.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924587 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0341 GB avg-val = 117.17 avg-zval = 36.89 2024/11/23-15:29:12.308982 139957223556672 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266585.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:29:13.117029 139957223556672 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266585.sst): second pass time = 1.33's, 81.405'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 49.291'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3254.869'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.340'MB/sec write SST data time = 0.13's, 308.592'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 27850.318'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.405'MB/sec zip pipeline throughput = 81.405'MB/sec entries = 924587 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.17 avg-zval = 36.89 usrkeys = 924587 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.78 avg-zval = 36.89 TagRS{ kinds = RS_Key0_TagN bytes = 144560 } TagArray{ size = 741590 bytes = 2780976 } seq expand size = 5191130 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0341 GB dict = 1.08 MB all = 0.0399 GB } UnZip/Zip{ index = 10.4444 value = 3.1765 dict = 1.00 all = 3.3646 } Zip/UnZip{ index = 0.0957 value = 0.3148 dict = 1.00 all = 0.2972 } ---------------------------- total value len = 10.325620 GB avg = 0.160 KB (by entry num) total key len = 1.783263 GB avg = 0.028 KB total ukey len = 1.242479 GB avg = 0.016 KB total ukey num = 0.077780492 Billion total entry num = 0.064588221 Billion write speed all = 0.512200540 MB/sec (with seq num) write speed all = 0.490344108 MB/sec (without seq num) 2024/11/23-15:29:13.165217 139957223556672 [compaction_job.cc:1948] [default] [JOB 29323] Generated table #266585: 924587 keys, 39890760 bytes, temperature: kWarm 2024/11/23-15:29:13.165286 139957223556672 EVENT_LOG_v1 {"time_micros": 1732346953165251, "cf_name": "default", "job": 29323, "event": "table_file_creation", "file_number": 266585, "file_size": 39890760, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012774830, "table_properties": {"data_size": 34103600, "index_size": 1770496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25888436, "raw_average_key_size": 28, "raw_value_size": 108329314, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924587, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346949, "oldest_key_time": 0, "file_creation_time": 1732346949, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266585, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:29:13.212818 139957223556672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266586.sst) /dev/shm/ToplingTemp/Topling-94870-uvQGIN 2024/11/23-15:29:14.065502 139957223556672 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266586.sst): first pass time = 0.85's, 157.413'MB/sec 2024/11/23-15:29:15.406355 139957223556672 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266586.sst): old prealloc_size = 147639500, real_size = 32644896 2024/11/23-15:29:15.406908 139957223556672 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266586.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932637 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0306 GB avg-val = 115.91 avg-zval = 32.85 2024/11/23-15:29:15.409002 139957223556672 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266586.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:29:16.263634 139957223556672 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266586.sst): second pass time = 1.34's, 80.679'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.017'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3330.809'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.13's, 289.567'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33573.156'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.679'MB/sec zip pipeline throughput = 80.679'MB/sec entries = 932637 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.91 avg-zval = 32.85 usrkeys = 932637 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.91 avg-zval = 32.85 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932637 bytes = 3380832 } seq expand size = 6528459 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5148 value = 3.5285 dict = 1.00 all = 3.6397 } Zip/UnZip{ index = 0.0951 value = 0.2834 dict = 1.00 all = 0.2747 } ---------------------------- total value len = 10.440253 GB avg = 0.159 KB (by entry num) total key len = 1.809377 GB avg = 0.028 KB total ukey len = 1.261132 GB avg = 0.016 KB total ukey num = 0.078713129 Billion total entry num = 0.065520858 Billion write speed all = 0.518085087 MB/sec (with seq num) write speed all = 0.495916005 MB/sec (without seq num) 2024/11/23-15:29:16.308780 139957223556672 [compaction_job.cc:1948] [default] [JOB 29323] Generated table #266586: 932637 keys, 36875916 bytes, temperature: kWarm 2024/11/23-15:29:16.308845 139957223556672 EVENT_LOG_v1 {"time_micros": 1732346956308810, "cf_name": "default", "job": 29323, "event": "table_file_creation", "file_number": 266586, "file_size": 36875916, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776228, "table_properties": {"data_size": 30637584, "index_size": 1773952, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26113836, "raw_average_key_size": 28, "raw_value_size": 108103910, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932637, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346953, "oldest_key_time": 0, "file_creation_time": 1732346953, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266586, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:29:16.349165 139957223556672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266587.sst) /dev/shm/ToplingTemp/Topling-94870-NbmlUP 2024/11/23-15:29:17.375247 139957223556672 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266587.sst): first pass time = 1.03's, 130.811'MB/sec 2024/11/23-15:29:18.702405 139957223556672 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266587.sst): old prealloc_size = 147639500, real_size = 31899040 2024/11/23-15:29:18.703020 139957223556672 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266587.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 936531 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1080 GB zip-val = 0.0299 GB avg-val = 115.31 avg-zval = 31.90 2024/11/23-15:29:18.705066 139957223556672 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266587.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:29:19.563240 139957223556672 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266587.sst): second pass time = 1.33's, 81.419'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 41.986'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3027.799'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.321'MB/sec write SST data time = 0.13's, 277.107'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 30954.511'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.419'MB/sec zip pipeline throughput = 81.419'MB/sec entries = 936531 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.31 avg-zval = 31.90 usrkeys = 936531 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.31 avg-zval = 31.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 936531 bytes = 3394944 } seq expand size = 6555717 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.4445 value = 3.6153 dict = 1.00 all = 3.7136 } Zip/UnZip{ index = 0.0957 value = 0.2766 dict = 1.00 all = 0.2693 } ---------------------------- total value len = 10.554803 GB avg = 0.159 KB (by entry num) total key len = 1.835599 GB avg = 0.028 KB total ukey len = 1.279863 GB avg = 0.016 KB total ukey num = 0.079649660 Billion total entry num = 0.066457389 Billion write speed all = 0.523965832 MB/sec (with seq num) write speed all = 0.501483011 MB/sec (without seq num) 2024/11/23-15:29:19.608157 139957223556672 [compaction_job.cc:1948] [default] [JOB 29323] Generated table #266587: 936531 keys, 36142280 bytes, temperature: kWarm 2024/11/23-15:29:19.608230 139957223556672 EVENT_LOG_v1 {"time_micros": 1732346959608193, "cf_name": "default", "job": 29323, "event": "table_file_creation", "file_number": 266587, "file_size": 36142280, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776248, "table_properties": {"data_size": 29871376, "index_size": 1793344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26222868, "raw_average_key_size": 28, "raw_value_size": 107994926, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 936531, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346956, "oldest_key_time": 0, "file_creation_time": 1732346956, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266587, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:29:19.654758 139957223556672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266588.sst) /dev/shm/ToplingTemp/Topling-94870-OgzvFO 2024/11/23-15:29:20.706264 139957223556672 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266588.sst): first pass time = 1.05's, 127.649'MB/sec 2024/11/23-15:29:22.040799 139957223556672 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266588.sst): old prealloc_size = 147639500, real_size = 35057792 2024/11/23-15:29:22.041388 139957223556672 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266588.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925568 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0331 GB avg-val = 117.01 avg-zval = 35.71 2024/11/23-15:29:22.042917 139957223556672 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266588.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:29:22.883086 139957223556672 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266588.sst): second pass time = 1.33's, 81.260'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.759'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3143.054'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.324'MB/sec write SST data time = 0.13's, 308.581'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32596.429'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.260'MB/sec zip pipeline throughput = 81.260'MB/sec entries = 925568 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.01 avg-zval = 35.71 usrkeys = 925568 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.01 avg-zval = 35.71 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925568 bytes = 3470896 } seq expand size = 6478976 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0331 GB dict = 1.08 MB all = 0.0394 GB } UnZip/Zip{ index = 10.4434 value = 3.2765 dict = 1.00 all = 3.4079 } Zip/UnZip{ index = 0.0958 value = 0.3052 dict = 1.00 all = 0.2934 } ---------------------------- total value len = 10.669584 GB avg = 0.158 KB (by entry num) total key len = 1.861515 GB avg = 0.028 KB total ukey len = 1.298374 GB avg = 0.016 KB total ukey num = 0.080575228 Billion total entry num = 0.067382957 Billion write speed all = 0.529841235 MB/sec (with seq num) write speed all = 0.507048490 MB/sec (without seq num) 2024/11/23-15:29:22.930743 139957223556672 [compaction_job.cc:1948] [default] [JOB 29323] Generated table #266588: 925568 keys, 39383758 bytes, temperature: kWarm 2024/11/23-15:29:22.930816 139957223556672 EVENT_LOG_v1 {"time_micros": 1732346962930778, "cf_name": "default", "job": 29323, "event": "table_file_creation", "file_number": 266588, "file_size": 39383758, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776268, "table_properties": {"data_size": 33053680, "index_size": 1772544, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25915904, "raw_average_key_size": 28, "raw_value_size": 108301824, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925568, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346959, "oldest_key_time": 0, "file_creation_time": 1732346959, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266588, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:29:22.967638 139957223556672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266589.sst) /dev/shm/ToplingTemp/Topling-94870-nssFmQ 2024/11/23-15:29:23.994677 139957223556672 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266589.sst): first pass time = 1.03's, 130.689'MB/sec 2024/11/23-15:29:25.289068 139957223556672 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266589.sst): old prealloc_size = 147639500, real_size = 32056784 2024/11/23-15:29:25.289655 139957223556672 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266589.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931242 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0300 GB avg-val = 116.13 avg-zval = 32.26 2024/11/23-15:29:25.291803 139957223556672 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266589.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:29:26.133151 139957223556672 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266589.sst): second pass time = 1.29's, 83.601'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.988'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3173.782'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.326'MB/sec write SST data time = 0.13's, 282.649'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36441.561'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.601'MB/sec zip pipeline throughput = 83.601'MB/sec entries = 931242 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.13 avg-zval = 32.26 usrkeys = 931242 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.13 avg-zval = 32.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931242 bytes = 3375760 } seq expand size = 6518694 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.4429 value = 3.5999 dict = 1.00 all = 3.6991 } Zip/UnZip{ index = 0.0958 value = 0.2778 dict = 1.00 all = 0.2703 } ---------------------------- total value len = 10.784246 GB avg = 0.158 KB (by entry num) total key len = 1.887590 GB avg = 0.028 KB total ukey len = 1.316999 GB avg = 0.016 KB total ukey num = 0.081506470 Billion total entry num = 0.068314199 Billion write speed all = 0.535718250 MB/sec (with seq num) write speed all = 0.512613681 MB/sec (without seq num) 2024/11/23-15:29:26.177179 139957223556672 [compaction_job.cc:1948] [default] [JOB 29323] Generated table #266589: 931242 keys, 36284072 bytes, temperature: kWarm 2024/11/23-15:29:26.177250 139957223556672 EVENT_LOG_v1 {"time_micros": 1732346966177214, "cf_name": "default", "job": 29323, "event": "table_file_creation", "file_number": 266589, "file_size": 36284072, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776300, "table_properties": {"data_size": 30040320, "index_size": 1783488, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26074776, "raw_average_key_size": 28, "raw_value_size": 108142972, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931242, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346962, "oldest_key_time": 0, "file_creation_time": 1732346962, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266589, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:29:26.212703 139957223556672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266590.sst) /dev/shm/ToplingTemp/Topling-94870-TYxU7M 2024/11/23-15:29:27.270429 139957223556672 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266590.sst): first pass time = 1.06's, 126.897'MB/sec 2024/11/23-15:29:28.595915 139957223556672 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266590.sst): old prealloc_size = 147639500, real_size = 35182832 2024/11/23-15:29:28.596478 139957223556672 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266590.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922697 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0332 GB avg-val = 117.46 avg-zval = 35.98 2024/11/23-15:29:28.598649 139957223556672 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266590.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:29:29.445321 139957223556672 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266590.sst): second pass time = 1.32's, 81.815'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 46.506'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3263.413'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 306.570'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 38116.144'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.815'MB/sec zip pipeline throughput = 81.815'MB/sec entries = 922697 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.46 avg-zval = 35.98 usrkeys = 922697 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.46 avg-zval = 35.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922697 bytes = 3460128 } seq expand size = 6458879 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.5131 value = 3.2649 dict = 1.00 all = 3.3978 } Zip/UnZip{ index = 0.0951 value = 0.3063 dict = 1.00 all = 0.2943 } ---------------------------- total value len = 10.899087 GB avg = 0.157 KB (by entry num) total key len = 1.913426 GB avg = 0.028 KB total ukey len = 1.335453 GB avg = 0.016 KB total ukey num = 0.082429167 Billion total entry num = 0.069236896 Billion write speed all = 0.541589702 MB/sec (with seq num) write speed all = 0.518176346 MB/sec (without seq num) 2024/11/23-15:29:29.492917 139957223556672 [compaction_job.cc:1948] [default] [JOB 29323] Generated table #266590: 922697 keys, 39501122 bytes, temperature: kWarm 2024/11/23-15:29:29.493006 139957223556672 EVENT_LOG_v1 {"time_micros": 1732346969492953, "cf_name": "default", "job": 29323, "event": "table_file_creation", "file_number": 266590, "file_size": 39501122, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776316, "table_properties": {"data_size": 33196640, "index_size": 1755328, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25835516, "raw_average_key_size": 28, "raw_value_size": 108382282, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922697, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346966, "oldest_key_time": 0, "file_creation_time": 1732346966, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266590, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:29:29.536830 139957223556672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266591.sst) /dev/shm/ToplingTemp/Topling-94870-XoNmdN 2024/11/23-15:29:30.576921 139957223556672 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266591.sst): first pass time = 1.04's, 129.049'MB/sec 2024/11/23-15:29:31.888789 139957223556672 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266591.sst): old prealloc_size = 147639500, real_size = 32765024 2024/11/23-15:29:31.889379 139957223556672 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266591.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931083 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0307 GB avg-val = 116.15 avg-zval = 33.03 2024/11/23-15:29:31.891531 139957223556672 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266591.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:29:32.746203 139957223556672 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266591.sst): second pass time = 1.31's, 82.489'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.280'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3151.894'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 290.504'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34753.158'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.489'MB/sec zip pipeline throughput = 82.489'MB/sec entries = 931083 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.15 avg-zval = 33.03 usrkeys = 931083 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.15 avg-zval = 33.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931083 bytes = 3491584 } seq expand size = 6517581 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.4441 value = 3.5171 dict = 1.00 all = 3.6170 } Zip/UnZip{ index = 0.0957 value = 0.2843 dict = 1.00 all = 0.2765 } ---------------------------- total value len = 11.013752 GB avg = 0.157 KB (by entry num) total key len = 1.939496 GB avg = 0.028 KB total ukey len = 1.354074 GB avg = 0.016 KB total ukey num = 0.083360250 Billion total entry num = 0.070167979 Billion write speed all = 0.547462254 MB/sec (with seq num) write speed all = 0.523737350 MB/sec (without seq num) 2024/11/23-15:29:32.791452 139957223556672 [compaction_job.cc:1948] [default] [JOB 29323] Generated table #266591: 931083 keys, 37107980 bytes, temperature: kWarm 2024/11/23-15:29:32.791526 139957223556672 EVENT_LOG_v1 {"time_micros": 1732346972791489, "cf_name": "default", "job": 29323, "event": "table_file_creation", "file_number": 266591, "file_size": 37107980, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012776344, "table_properties": {"data_size": 30749072, "index_size": 1782976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26070324, "raw_average_key_size": 28, "raw_value_size": 108147522, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931083, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346969, "oldest_key_time": 0, "file_creation_time": 1732346969, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266591, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:29:32.833431 139957223556672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266592.sst) /dev/shm/ToplingTemp/Topling-94870-sxObKP 2024/11/23-15:29:33.847667 139957223556672 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266592.sst): first pass time = 1.01's, 132.339'MB/sec 2024/11/23-15:29:35.229522 139957223556672 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266592.sst): old prealloc_size = 147639500, real_size = 33834064 2024/11/23-15:29:35.230131 139957223556672 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266592.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927568 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0318 GB avg-val = 116.70 avg-zval = 34.31 2024/11/23-15:29:35.230722 139957223556672 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266592.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:29:36.033445 139957223556672 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266592.sst): second pass time = 1.38's, 78.380'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.80's, 46.097'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3038.455'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.343'MB/sec write SST data time = 0.13's, 291.223'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35766.634'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.380'MB/sec zip pipeline throughput = 78.380'MB/sec entries = 927568 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.70 avg-zval = 34.31 usrkeys = 927568 avg-key = 20.00 avg-zkey = 1.92 avg-val = 121.32 avg-zval = 34.31 TagRS{ kinds = RS_Key0_TagN bytes = 145040 } TagArray{ size = 612132 bytes = 2218992 } seq expand size = 4284924 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.4433 value = 3.4012 dict = 1.00 all = 3.6219 } Zip/UnZip{ index = 0.0958 value = 0.2940 dict = 1.00 all = 0.2761 } ---------------------------- total value len = 11.126283 GB avg = 0.156 KB (by entry num) total key len = 1.965468 GB avg = 0.028 KB total ukey len = 1.372626 GB avg = 0.016 KB total ukey num = 0.084287818 Billion total entry num = 0.071095547 Billion write speed all = 0.553239134 MB/sec (with seq num) write speed all = 0.529203944 MB/sec (without seq num) 2024/11/23-15:29:36.078961 139957223556672 [compaction_job.cc:1948] [default] [JOB 29323] Generated table #266592: 927568 keys, 37057358 bytes, temperature: kWarm 2024/11/23-15:29:36.079045 139957223556672 EVENT_LOG_v1 {"time_micros": 1732346976079010, "cf_name": "default", "job": 29323, "event": "table_file_creation", "file_number": 266592, "file_size": 37057358, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 31825600, "index_size": 1776384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25971904, "raw_average_key_size": 28, "raw_value_size": 108245864, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927568, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346972, "oldest_key_time": 0, "file_creation_time": 1732346972, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266592, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:29:36.116235 139957223556672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266593.sst) /dev/shm/ToplingTemp/Topling-94870-kRMCYQ 2024/11/23-15:29:37.122479 139957223556672 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266593.sst): first pass time = 1.01's, 133.390'MB/sec 2024/11/23-15:29:38.600528 139957223556672 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266593.sst): old prealloc_size = 147639500, real_size = 36827888 2024/11/23-15:29:38.601130 139957223556672 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266593.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926664 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0348 GB avg-val = 116.84 avg-zval = 37.59 2024/11/23-15:29:38.601373 139957223556672 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266593.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:29:39.312613 139957223556672 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266593.sst): second pass time = 1.48's, 73.291'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.71's, 53.806'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3046.755'MB/sec (index lex order gen) rebuild zvType time = 0.59's, 0.394'MB/sec write SST data time = 0.12's, 308.506'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33665.721'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.291'MB/sec zip pipeline throughput = 73.291'MB/sec entries = 926664 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.84 avg-zval = 37.59 usrkeys = 926664 avg-key = 20.00 avg-zkey = 1.90 avg-val = 118.01 avg-zval = 37.59 TagRS{ kinds = RS_Key0_TagN bytes = 144880 } TagArray{ size = 154398 bytes = 482512 } seq expand size = 1080786 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0348 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.5135 value = 3.1083 dict = 1.00 all = 3.5031 } Zip/UnZip{ index = 0.0951 value = 0.3217 dict = 1.00 all = 0.2855 } ---------------------------- total value len = 11.235635 GB avg = 0.156 KB (by entry num) total key len = 1.991414 GB avg = 0.028 KB total ukey len = 1.391159 GB avg = 0.016 KB total ukey num = 0.085214482 Billion total entry num = 0.072022211 Billion write speed all = 0.558879218 MB/sec (with seq num) write speed all = 0.534534126 MB/sec (without seq num) 2024/11/23-15:29:39.359577 139957223556672 [compaction_job.cc:1948] [default] [JOB 29323] Generated table #266593: 926664 keys, 38313665 bytes, temperature: kWarm 2024/11/23-15:29:39.359657 139957223556672 EVENT_LOG_v1 {"time_micros": 1732346979359611, "cf_name": "default", "job": 29323, "event": "table_file_creation", "file_number": 266593, "file_size": 38313665, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700812, "table_properties": {"data_size": 34833248, "index_size": 1762816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25946592, "raw_average_key_size": 28, "raw_value_size": 108271212, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926664, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346976, "oldest_key_time": 0, "file_creation_time": 1732346976, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266593, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:29:39.399745 139957223556672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266594.sst) /dev/shm/ToplingTemp/Topling-94870-sFfKHO 2024/11/23-15:29:40.301721 139957223556672 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266594.sst): first pass time = 0.90's, 148.811'MB/sec 2024/11/23-15:29:41.638056 139957223556672 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266594.sst): old prealloc_size = 147639500, real_size = 35909680 2024/11/23-15:29:41.638612 139957223556672 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266594.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923571 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1084 GB zip-val = 0.0339 GB avg-val = 117.32 avg-zval = 36.74 2024/11/23-15:29:41.639234 139957223556672 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266594.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:29:42.442784 139957223556672 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266594.sst): second pass time = 1.34's, 81.166'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.80's, 48.740'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3161.680'MB/sec (index lex order gen) rebuild zvType time = 0.67's, 0.342'MB/sec write SST data time = 0.13's, 301.685'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 34371.321'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.166'MB/sec zip pipeline throughput = 81.166'MB/sec entries = 923571 avg-key = 28.00 avg-zkey = 1.89 avg-val = 117.32 avg-zval = 36.74 usrkeys = 923571 avg-key = 20.00 avg-zkey = 1.89 avg-val = 122.70 avg-zval = 36.74 TagRS{ kinds = RS_Key0_TagN bytes = 144400 } TagArray{ size = 709680 bytes = 2306480 } seq expand size = 4967760 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0339 GB dict = 1.08 MB all = 0.0392 GB } UnZip/Zip{ index = 10.5840 value = 3.1932 dict = 1.00 all = 3.4220 } Zip/UnZip{ index = 0.0945 value = 0.3132 dict = 1.00 all = 0.2922 } ---------------------------- total value len = 11.348960 GB avg = 0.156 KB (by entry num) total key len = 2.017274 GB avg = 0.028 KB total ukey len = 1.409630 GB avg = 0.016 KB total ukey num = 0.086138053 Billion total entry num = 0.072945782 Billion write speed all = 0.564685505 MB/sec (with seq num) write speed all = 0.540031486 MB/sec (without seq num) 2024/11/23-15:29:42.490317 139957223556672 [compaction_job.cc:1948] [default] [JOB 29323] Generated table #266594: 923571 keys, 39221736 bytes, temperature: kWarm 2024/11/23-15:29:42.490388 139957223556672 EVENT_LOG_v1 {"time_micros": 1732346982490352, "cf_name": "default", "job": 29323, "event": "table_file_creation", "file_number": 266594, "file_size": 39221736, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4068341420, "table_properties": {"data_size": 33933408, "index_size": 1745216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25859988, "raw_average_key_size": 28, "raw_value_size": 108357778, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923571, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346979, "oldest_key_time": 0, "file_creation_time": 1732346979, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266594, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:29:42.530611 139957223556672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266595.sst) /dev/shm/ToplingTemp/Topling-94870-M2h6pQ 2024/11/23-15:29:43.545292 139957223556672 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266595.sst): first pass time = 1.01's, 132.283'MB/sec 2024/11/23-15:29:44.935309 139957223556672 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266595.sst): old prealloc_size = 147639500, real_size = 36797136 2024/11/23-15:29:44.935872 139957223556672 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266595.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927361 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0348 GB avg-val = 116.73 avg-zval = 37.53 2024/11/23-15:29:44.937193 139957223556672 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266595.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:29:45.750414 139957223556672 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266595.sst): second pass time = 1.39's, 77.953'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 49.892'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3263.721'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.338'MB/sec write SST data time = 0.13's, 317.946'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 36540.992'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 77.953'MB/sec zip pipeline throughput = 77.953'MB/sec entries = 927361 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.73 avg-zval = 37.53 usrkeys = 927361 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.73 avg-zval = 37.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927361 bytes = 3013936 } seq expand size = 6491527 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0348 GB dict = 1.08 MB all = 0.0407 GB } UnZip/Zip{ index = 10.5111 value = 3.1106 dict = 1.00 all = 3.3005 } Zip/UnZip{ index = 0.0951 value = 0.3215 dict = 1.00 all = 0.3030 } ---------------------------- total value len = 11.463704 GB avg = 0.155 KB (by entry num) total key len = 2.043240 GB avg = 0.028 KB total ukey len = 1.428178 GB avg = 0.016 KB total ukey num = 0.087065414 Billion total entry num = 0.073873143 Billion write speed all = 0.570550348 MB/sec (with seq num) write speed all = 0.545586390 MB/sec (without seq num) 2024/11/23-15:29:45.799607 139957223556672 [compaction_job.cc:1948] [default] [JOB 29323] Generated table #266595: 927361 keys, 40665886 bytes, temperature: kWarm 2024/11/23-15:29:45.799688 139957223556672 EVENT_LOG_v1 {"time_micros": 1732346985799651, "cf_name": "default", "job": 29323, "event": "table_file_creation", "file_number": 266595, "file_size": 40665886, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4031991524, "largest_seqno": 4068341356, "table_properties": {"data_size": 34800576, "index_size": 1764544, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25966108, "raw_average_key_size": 28, "raw_value_size": 108251686, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927361, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346982, "oldest_key_time": 0, "file_creation_time": 1732346982, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266595, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:29:45.841733 139957223556672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266596.sst) /dev/shm/ToplingTemp/Topling-94870-WOgWXO 2024/11/23-15:29:46.905168 139957223556672 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266596.sst): first pass time = 1.06's, 126.216'MB/sec 2024/11/23-15:29:48.206460 139957223556672 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266596.sst): old prealloc_size = 147639500, real_size = 36380864 2024/11/23-15:29:48.207076 139957223556672 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266596.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929955 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0344 GB avg-val = 116.33 avg-zval = 36.96 2024/11/23-15:29:48.207808 139957223556672 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266596.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:29:49.032378 139957223556672 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266596.sst): second pass time = 1.30's, 83.178'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 48.606'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3000.091'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 307.820'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 31794.896'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.178'MB/sec zip pipeline throughput = 83.178'MB/sec entries = 929955 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.33 avg-zval = 36.96 usrkeys = 929955 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.33 avg-zval = 36.96 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929955 bytes = 2906128 } seq expand size = 6509685 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0401 GB } UnZip/Zip{ index = 10.4454 value = 3.1477 dict = 1.00 all = 3.3435 } Zip/UnZip{ index = 0.0957 value = 0.3177 dict = 1.00 all = 0.2991 } ---------------------------- total value len = 11.578392 GB avg = 0.155 KB (by entry num) total key len = 2.069279 GB avg = 0.028 KB total ukey len = 1.446777 GB avg = 0.016 KB total ukey num = 0.087995369 Billion total entry num = 0.074803098 Billion write speed all = 0.576414943 MB/sec (with seq num) write speed all = 0.551140230 MB/sec (without seq num) 2024/11/23-15:29:49.081259 139957223556672 [compaction_job.cc:1948] [default] [JOB 29323] Generated table #266596: 929955 keys, 40143008 bytes, temperature: kWarm 2024/11/23-15:29:49.081328 139957223556672 EVENT_LOG_v1 {"time_micros": 1732346989081293, "cf_name": "default", "job": 29323, "event": "table_file_creation", "file_number": 266596, "file_size": 40143008, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4042068720, "largest_seqno": 4068341454, "table_properties": {"data_size": 34367600, "index_size": 1780608, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26038740, "raw_average_key_size": 28, "raw_value_size": 108179058, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929955, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346985, "oldest_key_time": 0, "file_creation_time": 1732346985, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266596, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:29:49.121860 139957223556672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266597.sst) /dev/shm/ToplingTemp/Topling-94870-NhKrIP 2024/11/23-15:29:49.265826 139957223556672 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266597.sst): first pass time = 0.14's, 238.206'MB/sec 2024/11/23-15:29:49.589016 139957223556672 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266597.sst): old prealloc_size = 147639500, real_size = 9905584 2024/11/23-15:29:49.589232 139957223556672 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266597.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 238430 prefix = 4 raw-key = 0.0038 GB zip-key = 0.0005 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0276 GB zip-val = 0.0094 GB avg-val = 115.79 avg-zval = 39.36 2024/11/23-15:29:49.589462 139957223556672 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266597.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.034 GB 2024/11/23-15:29:49.793786 139957223556672 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00/266597.sst): second pass time = 0.32's, 85.606'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.20's, 53.117'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2537.872'MB/sec (index lex order gen) rebuild zvType time = 0.17's, 0.348'MB/sec write SST data time = 0.03's, 326.621'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.283'MB, throughput = 8767.021'MB/sec zip my value time = 0.32's, unzip length = 0.028'GB zip my value throughput = 85.606'MB/sec zip pipeline throughput = 85.606'MB/sec entries = 238430 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.79 avg-zval = 39.36 usrkeys = 238430 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.79 avg-zval = 39.36 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 238430 bytes = 745104 } seq expand size = 1669010 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0048 GB value = 0.0276 GB dict = 0.28 MB all = 0.0343 GB } __ZipSize{ index = 0.0005 GB value = 0.0094 GB dict = 0.28 MB all = 0.0109 GB } UnZip/Zip{ index = 10.3514 value = 2.9416 dict = 1.00 all = 3.1520 } Zip/UnZip{ index = 0.0966 value = 0.3399 dict = 1.00 all = 0.3173 } ---------------------------- total value len = 11.607669 GB avg = 0.155 KB (by entry num) total key len = 2.075955 GB avg = 0.028 KB total ukey len = 1.451545 GB avg = 0.016 KB total ukey num = 0.088233799 Billion total entry num = 0.075041528 Billion write speed all = 0.577914835 MB/sec (with seq num) write speed all = 0.552560375 MB/sec (without seq num) 2024/11/23-15:29:49.812568 139957223556672 [compaction_job.cc:1948] [default] [JOB 29323] Generated table #266597: 238430 keys, 10876653 bytes, temperature: kWarm 2024/11/23-15:29:49.812641 139957223556672 EVENT_LOG_v1 {"time_micros": 1732346989812599, "cf_name": "default", "job": 29323, "event": "table_file_creation", "file_number": 266597, "file_size": 10876653, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4042106349, "largest_seqno": 4068341129, "table_properties": {"data_size": 9385120, "index_size": 460672, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 6676040, "raw_average_key_size": 28, "raw_value_size": 27607640, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 238430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346989, "oldest_key_time": 0, "file_creation_time": 1732346989, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266597, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:29:49.826529 139957223556672 [compaction_job.cc:703] job-29323: subcompact[0], size: 0.918989 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29323/att-00 [266573,266574,266575,266576,266577,266578,266579,266580,266581,266582,266583,266584,266585,266586,266587,266588,266589,266590,266591,266592,266593,266594,266595,266596,266597] 2024/11/23-15:29:49.972321 139957223556672 (Original Log Time 2024/11/23-15:28:26.461028) EVENT_LOG_v1 {"time_micros": 1732346906460973, "job": 29323, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [266541], "files_L3": [266567, 266568, 266496, 266497, 266498, 266499, 266500, 266501, 266502, 266503, 266504, 266505, 266506, 266507, 266508, 266509, 266510, 266511, 266512, 266513, 266514, 266515, 266516, 266517, 266569], "score": 0.99079, "input_data_size": 939534109, "oldest_snapshot_seqno": 4068547984} 2024/11/23-15:29:49.972325 139957223556672 (Original Log Time 2024/11/23-15:29:49.970423) [compaction_job.cc:2022] [default] [JOB 29323] Compacted 1@2 + 25@3 files to L3 => 918988593 bytes 2024/11/23-15:29:49.972326 139957223556672 (Original Log Time 2024/11/23-15:29:49.972246) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 25 0 0 0] max score 0.03, MB/sec: 11.3 rd, 11.0 wr, level 3, files in(1, 25) out(25 +0 blob) MB in(27.1, 868.9 +0.0 blob) out(876.4 +0.0 blob), read-write-amplify(65.3) write-amplify(32.3) OK, records in: 22760800, records dropped: 77436 output_compression: Snappy 2024/11/23-15:29:49.972330 139957223556672 (Original Log Time 2024/11/23-15:29:49.972293) EVENT_LOG_v1 {"time_micros": 1732346989972270, "cf": "default", "job": 29323, "event": "compaction_finished", "compaction_time_micros": 83365509, "compaction_time_cpu_micros": 35687850, "output_level": 3, "num_output_files": 25, "total_output_size": 918988593, "num_input_records": 22760800, "num_output_records": 22683364, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 684155480, "file_range_sync_nanos": 0, "file_fsync_nanos": 17496, "file_prepare_write_nanos": 71428, "lsm_state": [0, 0, 0, 25, 0, 0, 0]} 2024/11/23-15:29:50.121198 139957223556672 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732346906,"executesMs":83668,"compactionJobId":29323,"attempt":0,"compactionInputRawBytes":3266506673,"compactionInputZipBytes":939534109,"compactionOutputRawBytes":0,"compactionOutputZipBytes":918988593,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-d45cbcaaec31a8d6b344ef59e50f4cc9-2dbb605625ab41d5-00"}}, response = 2024/11/23-15:29:50.121287 139957223556672 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-29323/att-00: olev 3, work 83.668 s, result 7.702 ms, install 1.926 ms, input{raw 3.042 GiB zip 896.010 MiB}