2024/11/21-00:05:41.310136 140008710874688 DEBG dcompact_worker.cpp:2219: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00: fork to child time = 0.007235 sec 2024/11/21-00:05:41.310542 140008710874688 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00 2024/11/21-00:05:41.316705 140008710874688 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00 2024/11/21-00:05:41.318372 140008710874688 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/21-00:05:41.318522 140008710874688 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/21-00:05:41.364339 140008710874688 [version_set.cc:6211] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-22223/att-00/MANIFEST-049091 2024/11/21-00:05:41.440546 140008710874688 [version_set.cc:6254] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-22223/att-00/MANIFEST-049091 succeeded,manifest_file_number is 49091, next_file_number is 88042, last_sequence is 1343995954, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 87852 2024/11/21-00:05:41.440552 140008710874688 [version_set.cc:6269] Column family [default] (ID 0), log number is 0 2024/11/21-00:05:41.440629 140008710874688 [version_set.cc:5724] Creating manifest 88042 2024/11/21-00:05:41.474488 140008710874688 DEBG dcompact_worker.cpp:1225: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/21-00:05:41.496443 140008710874688 [compaction_job.cc:2437] [default] [JOB 22223] Compacting 1@2 + 20@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/21-00:05:41.496454 140008710874688 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 16600 Base level 2, inputs: [87982(27MB)], [88009(37MB) 88010(3243KB) 87930(37MB) 87931(37MB) 87932(36MB) 87933(40MB) 87934(37MB) 87935(38MB) 87936(38MB) 87937(38MB) 87938(36MB) 87939(36MB) 87942(36MB) 87943(37MB) 87944(37MB) 87945(36MB) 87946(37MB) 87947(37MB) 87948(40MB) 88029(35MB)] 2024/11/21-00:05:41.505843 140008710874688 [top_zip_table.cc:59] core git_version_hash_info_is:commit 81964c21ebbcc178d8e99dae0fbf61bded2729cc Author: rockeet Date: Mon Nov 11 21:28:46 2024 +0800 Makefile: fix gen git-version-*.cpp GIT_PATH_ARG = :!src/terark/fsa :!tools/fsa :!src/terark/zbs :!tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/21-00:05:41.505862 140008710874688 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit 5dfecb7d107196f044ff5bf78dc39d2e725b9d9b Author: leipeng Date: Thu Nov 7 15:22:45 2024 +0800 Makefile: fix for x86_64 CPU older than haswell GIT_PATH_ARG = src/terark/fsa tools/fsa g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/21-00:05:41.505878 140008710874688 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit d44dd5ff72707128fe3c916077b43d2093c01f7c Author: leipeng Date: Sun Oct 27 23:38:26 2024 +0800 Fix for msvc, when toplingdb on msvc GIT_PATH_ARG = src/terark/zbs tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/21-00:05:41.505884 140008710874688 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit 79614a7e7dc27362fc5151b0797eb7d83c674c7b Author: rockeet Date: Sun Nov 10 15:02:21 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/21-00:05:41.505894 140008710874688 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit c46b65a52d9c74aa8b87aeb8438620ff2dbc669c Author: rockeet Date: Sun Nov 10 15:02:42 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/21-00:05:42.190557 140008710874688 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088043.sst) /dev/shm/ToplingTemp/Topling-1176197-ouI0xH 2024/11/21-00:05:42.689153 140008710874688 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088043.sst): first pass time = 0.50's, 269.215'MB/sec 2024/11/21-00:05:43.914817 140008710874688 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088043.sst): old prealloc_size = 147639500, real_size = 34717424 2024/11/21-00:05:43.915461 140008710874688 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088043.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926300 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0327 GB avg-val = 116.90 avg-zval = 35.33 2024/11/21-00:05:43.916881 140008710874688 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088043.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:05:44.761971 140008710874688 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088043.sst): second pass time = 1.22's, 88.407'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.818'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2817.761'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 307.817'MB/sec dict compress time = 3616740576.92's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 14999.653'MB/sec zip my value time = 1.22's, unzip length = 0.108'GB zip my value throughput = 88.407'MB/sec zip pipeline throughput = 88.407'MB/sec entries = 926300 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.90 avg-zval = 35.33 usrkeys = 926300 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.90 avg-zval = 35.33 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926300 bytes = 3242064 } seq expand size = 6484100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0327 GB dict = 1.08 MB all = 0.0388 GB } UnZip/Zip{ index = 10.5135 value = 3.3090 dict = 1.00 all = 3.4580 } Zip/UnZip{ index = 0.0951 value = 0.3022 dict = 1.00 all = 0.2892 } ---------------------------- total value len = 563.180498 GB avg = 0.331 KB (by entry num) total key len = 44.340343 GB avg = 0.026 KB total ukey len = 32.812668 GB avg = 0.014 KB total ukey num = 2.321940876 Billion total entry num = 1.701442898 Billion write speed all = 27.967465165 MB/sec (with seq num) write speed all = 27.340852324 MB/sec (without seq num) 2024/11/21-00:05:44.813522 140008710874688 [compaction_job.cc:1948] [default] [JOB 22223] Generated table #88043: 926300 keys, 38813826 bytes, temperature: kWarm 2024/11/21-00:05:44.813607 140008710874688 EVENT_LOG_v1 {"time_micros": 1732118744813567, "cf_name": "default", "job": 22223, "event": "table_file_creation", "file_number": 88043, "file_size": 38813826, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1156808633, "largest_seqno": 1343541316, "table_properties": {"data_size": 32723552, "index_size": 1762112, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25936400, "raw_average_key_size": 28, "raw_value_size": 108281440, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118742, "oldest_key_time": 0, "file_creation_time": 1732118742, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 88043, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:05:44.849278 140008710874688 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088044.sst) /dev/shm/ToplingTemp/Topling-1176197-tVnygI 2024/11/21-00:05:45.744937 140008710874688 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088044.sst): first pass time = 0.90's, 149.859'MB/sec 2024/11/21-00:05:46.937110 140008710874688 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088044.sst): old prealloc_size = 147639500, real_size = 34701728 2024/11/21-00:05:46.937748 140008710874688 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088044.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930392 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.97 raw-val = 0.1082 GB zip-val = 0.0326 GB avg-val = 116.26 avg-zval = 35.08 2024/11/21-00:05:46.938599 140008710874688 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088044.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:05:47.775779 140008710874688 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088044.sst): second pass time = 1.19's, 90.798'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.378'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2960.821'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 291.783'MB/sec dict compress time = 504403711.75's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34951.847'MB/sec zip my value time = 1.19's, unzip length = 0.108'GB zip my value throughput = 90.798'MB/sec zip pipeline throughput = 90.798'MB/sec entries = 930392 avg-key = 28.00 avg-zkey = 1.97 avg-val = 116.26 avg-zval = 35.08 usrkeys = 930392 avg-key = 20.00 avg-zkey = 1.97 avg-val = 122.88 avg-zval = 35.08 TagRS{ kinds = RS_Key0_TagN bytes = 145480 } TagArray{ size = 880542 bytes = 3191984 } seq expand size = 6163794 multi value expand size = 0 cnt WriteAppend = 84 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0326 GB dict = 1.08 MB all = 0.0389 GB } UnZip/Zip{ index = 10.1781 value = 3.3139 dict = 1.00 all = 3.4508 } Zip/UnZip{ index = 0.0983 value = 0.3018 dict = 1.00 all = 0.2898 } ---------------------------- total value len = 563.294828 GB avg = 0.331 KB (by entry num) total key len = 44.366394 GB avg = 0.026 KB total ukey len = 32.831276 GB avg = 0.014 KB total ukey num = 2.322871268 Billion total entry num = 1.702373290 Billion write speed all = 27.970047069 MB/sec (with seq num) write speed all = 27.343178553 MB/sec (without seq num) 2024/11/21-00:05:47.824383 140008710874688 [compaction_job.cc:1948] [default] [JOB 22223] Generated table #88044: 930392 keys, 38894758 bytes, temperature: kWarm 2024/11/21-00:05:47.824446 140008710874688 EVENT_LOG_v1 {"time_micros": 1732118747824415, "cf_name": "default", "job": 22223, "event": "table_file_creation", "file_number": 88044, "file_size": 38894758, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1343541812, "table_properties": {"data_size": 32640720, "index_size": 1828224, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26050976, "raw_average_key_size": 28, "raw_value_size": 108166760, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930392, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118744, "oldest_key_time": 0, "file_creation_time": 1732118744, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 88044, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:05:47.860150 140008710874688 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088045.sst) /dev/shm/ToplingTemp/Topling-1176197-SPEzhF 2024/11/21-00:05:48.650878 140008710874688 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088045.sst): first pass time = 0.79's, 169.747'MB/sec 2024/11/21-00:05:50.042179 140008710874688 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088045.sst): old prealloc_size = 147639500, real_size = 36438048 2024/11/21-00:05:50.042765 140008710874688 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088045.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926545 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.97 raw-val = 0.1083 GB zip-val = 0.0344 GB avg-val = 116.86 avg-zval = 37.11 2024/11/21-00:05:50.043354 140008710874688 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088045.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:05:50.742764 140008710874688 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088045.sst): second pass time = 1.39's, 77.868'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.70's, 56.419'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3309.484'MB/sec (index lex order gen) rebuild zvType time = 0.57's, 0.405'MB/sec write SST data time = 0.13's, 309.324'MB/sec dict compress time = 3632272088.38's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 20938.814'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 77.868'MB/sec zip pipeline throughput = 77.868'MB/sec entries = 926545 avg-key = 28.00 avg-zkey = 1.97 avg-val = 116.86 avg-zval = 37.11 usrkeys = 926545 avg-key = 20.00 avg-zkey = 1.97 avg-val = 121.52 avg-zval = 37.11 TagRS{ kinds = RS_Key0_TagN bytes = 144880 } TagArray{ size = 617488 bytes = 2084032 } seq expand size = 4322416 multi value expand size = 0 cnt WriteAppend = 51 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.1766 value = 3.1489 dict = 1.00 all = 3.3957 } Zip/UnZip{ index = 0.0983 value = 0.3176 dict = 1.00 all = 0.2945 } ---------------------------- total value len = 563.407425 GB avg = 0.331 KB (by entry num) total key len = 44.392337 GB avg = 0.026 KB total ukey len = 32.849807 GB avg = 0.014 KB total ukey num = 2.323797813 Billion total entry num = 1.703299835 Billion write speed all = 27.972603793 MB/sec (with seq num) write speed all = 27.345479738 MB/sec (without seq num) 2024/11/21-00:05:50.792970 140008710874688 [compaction_job.cc:1948] [default] [JOB 22223] Generated table #88045: 926545 keys, 39525388 bytes, temperature: kWarm 2024/11/21-00:05:50.793029 140008710874688 EVENT_LOG_v1 {"time_micros": 1732118750793000, "cf_name": "default", "job": 22223, "event": "table_file_creation", "file_number": 88045, "file_size": 39525388, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 978567417, "table_properties": {"data_size": 34385296, "index_size": 1820928, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25943260, "raw_average_key_size": 28, "raw_value_size": 108274606, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926545, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118747, "oldest_key_time": 0, "file_creation_time": 1732118747, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 88045, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:05:50.826257 140008710874688 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088046.sst) /dev/shm/ToplingTemp/Topling-1176197-0O5LPG 2024/11/21-00:05:51.588131 140008710874688 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088046.sst): first pass time = 0.76's, 176.175'MB/sec 2024/11/21-00:05:52.896667 140008710874688 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088046.sst): old prealloc_size = 147639500, real_size = 36425312 2024/11/21-00:05:52.897277 140008710874688 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088046.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926819 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.97 raw-val = 0.1083 GB zip-val = 0.0344 GB avg-val = 116.82 avg-zval = 37.09 2024/11/21-00:05:52.897450 140008710874688 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088046.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:05:53.593988 140008710874688 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088046.sst): second pass time = 1.31's, 82.789'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.70's, 53.912'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3090.166'MB/sec (index lex order gen) rebuild zvType time = 0.57's, 0.406'MB/sec write SST data time = 0.13's, 297.714'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34220.414'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.789'MB/sec zip pipeline throughput = 82.789'MB/sec entries = 926819 avg-key = 28.00 avg-zkey = 1.97 avg-val = 116.82 avg-zval = 37.09 usrkeys = 926819 avg-key = 20.00 avg-zkey = 1.97 avg-val = 117.18 avg-zval = 37.09 TagRS{ kinds = RS_Key0_TagN bytes = 144920 } TagArray{ size = 48570 bytes = 163936 } seq expand size = 339990 multi value expand size = 0 cnt WriteAppend = 40 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.1743 value = 3.1499 dict = 1.00 all = 3.5703 } Zip/UnZip{ index = 0.0983 value = 0.3175 dict = 1.00 all = 0.2801 } ---------------------------- total value len = 563.516032 GB avg = 0.331 KB (by entry num) total key len = 44.418288 GB avg = 0.026 KB total ukey len = 32.868343 GB avg = 0.014 KB total ukey num = 2.324724632 Billion total entry num = 1.704226654 Billion write speed all = 27.975125579 MB/sec (with seq num) write speed all = 27.347742612 MB/sec (without seq num) 2024/11/21-00:05:53.645081 140008710874688 [compaction_job.cc:1948] [default] [JOB 22223] Generated table #88046: 926819 keys, 37592657 bytes, temperature: kWarm 2024/11/21-00:05:53.645137 140008710874688 EVENT_LOG_v1 {"time_micros": 1732118753645108, "cf_name": "default", "job": 22223, "event": "table_file_creation", "file_number": 88046, "file_size": 37592657, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 972340122, "table_properties": {"data_size": 34371536, "index_size": 1821888, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25950932, "raw_average_key_size": 28, "raw_value_size": 108266854, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926819, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118750, "oldest_key_time": 0, "file_creation_time": 1732118750, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 88046, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:05:53.678821 140008710874688 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088047.sst) /dev/shm/ToplingTemp/Topling-1176197-Evxy2G 2024/11/21-00:05:54.516104 140008710874688 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088047.sst): first pass time = 0.84's, 160.307'MB/sec 2024/11/21-00:05:55.748203 140008710874688 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088047.sst): old prealloc_size = 147639500, real_size = 38415552 2024/11/21-00:05:55.748823 140008710874688 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088047.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926575 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1083 GB zip-val = 0.0364 GB avg-val = 116.85 avg-zval = 39.27 2024/11/21-00:05:55.749674 140008710874688 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088047.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:05:56.519366 140008710874688 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088047.sst): second pass time = 1.23's, 87.944'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.77's, 55.288'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3176.293'MB/sec (index lex order gen) rebuild zvType time = 0.64's, 0.363'MB/sec write SST data time = 0.13's, 321.486'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 16162.804'MB/sec zip my value time = 1.23's, unzip length = 0.108'GB zip my value throughput = 87.944'MB/sec zip pipeline throughput = 87.944'MB/sec entries = 926575 avg-key = 28.00 avg-zkey = 1.94 avg-val = 116.85 avg-zval = 39.27 usrkeys = 926575 avg-key = 20.00 avg-zkey = 1.94 avg-val = 123.56 avg-zval = 39.27 TagRS{ kinds = RS_Key0_TagN bytes = 144880 } TagArray{ size = 887648 bytes = 3217744 } seq expand size = 6213536 multi value expand size = 0 cnt WriteAppend = 21 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0364 GB dict = 1.08 MB all = 0.0426 GB } UnZip/Zip{ index = 10.3110 value = 2.9757 dict = 1.00 all = 3.1480 } Zip/UnZip{ index = 0.0970 value = 0.3361 dict = 1.00 all = 0.3177 } ---------------------------- total value len = 563.630519 GB avg = 0.331 KB (by entry num) total key len = 44.444232 GB avg = 0.026 KB total ukey len = 32.886874 GB avg = 0.014 KB total ukey num = 2.325651207 Billion total entry num = 1.705153229 Billion write speed all = 27.977821500 MB/sec (with seq num) write speed all = 27.350181921 MB/sec (without seq num) 2024/11/21-00:05:56.577716 140008710874688 [compaction_job.cc:1948] [default] [JOB 22223] Generated table #88047: 926575 keys, 42636256 bytes, temperature: kWarm 2024/11/21-00:05:56.577788 140008710874688 EVENT_LOG_v1 {"time_micros": 1732118756577754, "cf_name": "default", "job": 22223, "event": "table_file_creation", "file_number": 88047, "file_size": 42636256, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1325590673, "table_properties": {"data_size": 36386480, "index_size": 1797248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25944100, "raw_average_key_size": 28, "raw_value_size": 108273690, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926575, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118753, "oldest_key_time": 0, "file_creation_time": 1732118753, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 88047, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:05:56.615458 140008710874688 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088048.sst) /dev/shm/ToplingTemp/Topling-1176197-DPnOhF 2024/11/21-00:05:57.471701 140008710874688 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088048.sst): first pass time = 0.86's, 156.760'MB/sec 2024/11/21-00:05:58.588856 140008710874688 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088048.sst): old prealloc_size = 147639500, real_size = 35103312 2024/11/21-00:05:58.589440 140008710874688 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088048.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925770 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0331 GB avg-val = 116.98 avg-zval = 35.77 2024/11/21-00:05:58.590230 140008710874688 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088048.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:05:59.438927 140008710874688 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088048.sst): second pass time = 1.12's, 97.011'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 46.113'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3161.448'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 307.684'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32259.488'MB/sec zip my value time = 1.12's, unzip length = 0.108'GB zip my value throughput = 97.011'MB/sec zip pipeline throughput = 97.011'MB/sec entries = 925770 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.98 avg-zval = 35.77 usrkeys = 925770 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.98 avg-zval = 35.77 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925770 bytes = 3240208 } seq expand size = 6480390 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0331 GB dict = 1.08 MB all = 0.0392 GB } UnZip/Zip{ index = 10.5136 value = 3.2707 dict = 1.00 all = 3.4241 } Zip/UnZip{ index = 0.0951 value = 0.3057 dict = 1.00 all = 0.2920 } ---------------------------- total value len = 563.745296 GB avg = 0.330 KB (by entry num) total key len = 44.470154 GB avg = 0.026 KB total ukey len = 32.905390 GB avg = 0.014 KB total ukey num = 2.326576977 Billion total entry num = 1.706078999 Billion write speed all = 27.980536459 MB/sec (with seq num) write speed all = 27.352640464 MB/sec (without seq num) 2024/11/21-00:05:59.487089 140008710874688 [compaction_job.cc:1948] [default] [JOB 22223] Generated table #88048: 925770 keys, 39198242 bytes, temperature: kWarm 2024/11/21-00:05:59.487161 140008710874688 EVENT_LOG_v1 {"time_micros": 1732118759487128, "cf_name": "default", "job": 22223, "event": "table_file_creation", "file_number": 88048, "file_size": 39198242, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1187855341, "largest_seqno": 1331717282, "table_properties": {"data_size": 33110592, "index_size": 1761088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25921560, "raw_average_key_size": 28, "raw_value_size": 108296304, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925770, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118756, "oldest_key_time": 0, "file_creation_time": 1732118756, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 88048, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:05:59.527801 140008710874688 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088049.sst) /dev/shm/ToplingTemp/Topling-1176197-jxLP5E 2024/11/21-00:06:00.542295 140008710874688 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088049.sst): first pass time = 1.01's, 132.306'MB/sec 2024/11/21-00:06:01.678701 140008710874688 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088049.sst): old prealloc_size = 147639500, real_size = 35975296 2024/11/21-00:06:01.679282 140008710874688 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088049.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928214 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0340 GB avg-val = 116.60 avg-zval = 36.61 2024/11/21-00:06:01.680095 140008710874688 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088049.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:06:02.502527 140008710874688 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088049.sst): second pass time = 1.14's, 95.311'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 48.648'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3241.969'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.333'MB/sec write SST data time = 0.13's, 316.154'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 15573.955'MB/sec zip my value time = 1.14's, unzip length = 0.108'GB zip my value throughput = 95.311'MB/sec zip pipeline throughput = 95.311'MB/sec entries = 928214 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.60 avg-zval = 36.61 usrkeys = 928214 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.60 avg-zval = 36.61 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928214 bytes = 3248768 } seq expand size = 6497498 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0340 GB dict = 1.08 MB all = 0.0401 GB } UnZip/Zip{ index = 10.5139 value = 3.1853 dict = 1.00 all = 3.3489 } Zip/UnZip{ index = 0.0951 value = 0.3139 dict = 1.00 all = 0.2986 } ---------------------------- total value len = 563.860021 GB avg = 0.330 KB (by entry num) total key len = 44.496144 GB avg = 0.026 KB total ukey len = 32.923954 GB avg = 0.014 KB total ukey num = 2.327505191 Billion total entry num = 1.707007213 Billion write speed all = 27.983066071 MB/sec (with seq num) write speed all = 27.354916991 MB/sec (without seq num) 2024/11/21-00:06:02.554293 140008710874688 [compaction_job.cc:1948] [default] [JOB 22223] Generated table #88049: 928214 keys, 40077584 bytes, temperature: kWarm 2024/11/21-00:06:02.554365 140008710874688 EVENT_LOG_v1 {"time_micros": 1732118762554332, "cf_name": "default", "job": 22223, "event": "table_file_creation", "file_number": 88049, "file_size": 40077584, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1187855426, "largest_seqno": 1331716698, "table_properties": {"data_size": 33977392, "index_size": 1765696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25989992, "raw_average_key_size": 28, "raw_value_size": 108227824, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928214, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118759, "oldest_key_time": 0, "file_creation_time": 1732118759, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 88049, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:06:02.589270 140008710874688 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088050.sst) /dev/shm/ToplingTemp/Topling-1176197-glQO9G 2024/11/21-00:06:03.622303 140008710874688 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088050.sst): first pass time = 1.03's, 129.930'MB/sec 2024/11/21-00:06:04.769657 140008710874688 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088050.sst): old prealloc_size = 147639500, real_size = 36101264 2024/11/21-00:06:04.770236 140008710874688 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088050.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928488 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0341 GB avg-val = 116.56 avg-zval = 36.73 2024/11/21-00:06:04.771038 140008710874688 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088050.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:06:05.584793 140008710874688 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088050.sst): second pass time = 1.15's, 94.393'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 49.327'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3196.312'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.337'MB/sec write SST data time = 0.13's, 319.072'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33342.107'MB/sec zip my value time = 1.15's, unzip length = 0.108'GB zip my value throughput = 94.393'MB/sec zip pipeline throughput = 94.393'MB/sec entries = 928488 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.56 avg-zval = 36.73 usrkeys = 928488 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.56 avg-zval = 36.73 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928488 bytes = 3249728 } seq expand size = 6499416 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0341 GB dict = 1.08 MB all = 0.0402 GB } UnZip/Zip{ index = 10.5124 value = 3.1734 dict = 1.00 all = 3.3381 } Zip/UnZip{ index = 0.0951 value = 0.3151 dict = 1.00 all = 0.2996 } ---------------------------- total value len = 563.974741 GB avg = 0.330 KB (by entry num) total key len = 44.522141 GB avg = 0.026 KB total ukey len = 32.942524 GB avg = 0.014 KB total ukey num = 2.328433679 Billion total entry num = 1.707935701 Billion write speed all = 27.985571029 MB/sec (with seq num) write speed all = 27.357169376 MB/sec (without seq num) 2024/11/21-00:06:05.635511 140008710874688 [compaction_job.cc:1948] [default] [JOB 22223] Generated table #88050: 928488 keys, 40207620 bytes, temperature: kWarm 2024/11/21-00:06:05.635574 140008710874688 EVENT_LOG_v1 {"time_micros": 1732118765635543, "cf_name": "default", "job": 22223, "event": "table_file_creation", "file_number": 88050, "file_size": 40207620, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1187851486, "largest_seqno": 1331716806, "table_properties": {"data_size": 34102464, "index_size": 1766464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25997664, "raw_average_key_size": 28, "raw_value_size": 108220120, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928488, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118762, "oldest_key_time": 0, "file_creation_time": 1732118762, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 88050, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:06:05.677596 140008710874688 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088051.sst) /dev/shm/ToplingTemp/Topling-1176197-MvhbVG 2024/11/21-00:06:06.545864 140008710874688 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088051.sst): first pass time = 0.87's, 154.588'MB/sec 2024/11/21-00:06:07.693228 140008710874688 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088051.sst): old prealloc_size = 147639500, real_size = 36366368 2024/11/21-00:06:07.693797 140008710874688 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088051.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930123 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0344 GB avg-val = 116.30 avg-zval = 36.93 2024/11/21-00:06:07.694587 140008710874688 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088051.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:06:08.514850 140008710874688 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088051.sst): second pass time = 1.15's, 94.348'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 49.262'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3253.150'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.337'MB/sec write SST data time = 0.13's, 307.069'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 22203.786'MB/sec zip my value time = 1.15's, unzip length = 0.108'GB zip my value throughput = 94.348'MB/sec zip pipeline throughput = 94.348'MB/sec entries = 930123 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.30 avg-zval = 36.93 usrkeys = 930123 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.30 avg-zval = 36.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930123 bytes = 3255440 } seq expand size = 6510861 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0405 GB } UnZip/Zip{ index = 10.4439 value = 3.1490 dict = 1.00 all = 3.3161 } Zip/UnZip{ index = 0.0957 value = 0.3176 dict = 1.00 all = 0.3016 } ---------------------------- total value len = 564.089426 GB avg = 0.330 KB (by entry num) total key len = 44.548185 GB avg = 0.026 KB total ukey len = 32.961126 GB avg = 0.014 KB total ukey num = 2.329363802 Billion total entry num = 1.708865824 Billion write speed all = 27.988271712 MB/sec (with seq num) write speed all = 27.359612554 MB/sec (without seq num) 2024/11/21-00:06:08.567729 140008710874688 [compaction_job.cc:1948] [default] [JOB 22223] Generated table #88051: 930123 keys, 40474192 bytes, temperature: kWarm 2024/11/21-00:06:08.567786 140008710874688 EVENT_LOG_v1 {"time_micros": 1732118768567755, "cf_name": "default", "job": 22223, "event": "table_file_creation", "file_number": 88051, "file_size": 40474192, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1187851565, "largest_seqno": 1331717093, "table_properties": {"data_size": 34352464, "index_size": 1781184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26043444, "raw_average_key_size": 28, "raw_value_size": 108174326, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930123, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118765, "oldest_key_time": 0, "file_creation_time": 1732118765, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 88051, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:06:08.603448 140008710874688 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088052.sst) /dev/shm/ToplingTemp/Topling-1176197-ldv8WI 2024/11/21-00:06:09.615355 140008710874688 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088052.sst): first pass time = 1.01's, 132.643'MB/sec 2024/11/21-00:06:10.768773 140008710874688 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088052.sst): old prealloc_size = 147639500, real_size = 33925088 2024/11/21-00:06:10.769365 140008710874688 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088052.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926933 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0319 GB avg-val = 116.80 avg-zval = 34.46 2024/11/21-00:06:10.770194 140008710874688 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088052.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:06:11.615823 140008710874688 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088052.sst): second pass time = 1.15's, 93.932'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.031'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3083.474'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.13's, 292.043'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31267.643'MB/sec zip my value time = 1.15's, unzip length = 0.108'GB zip my value throughput = 93.932'MB/sec zip pipeline throughput = 93.932'MB/sec entries = 926933 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.80 avg-zval = 34.46 usrkeys = 926933 avg-key = 20.00 avg-zkey = 1.89 avg-val = 123.80 avg-zval = 34.46 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926933 bytes = 3360144 } seq expand size = 6488531 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.5826 value = 3.3894 dict = 1.00 all = 3.5188 } Zip/UnZip{ index = 0.0945 value = 0.2950 dict = 1.00 all = 0.2842 } ---------------------------- total value len = 564.204178 GB avg = 0.330 KB (by entry num) total key len = 44.574139 GB avg = 0.026 KB total ukey len = 32.979665 GB avg = 0.014 KB total ukey num = 2.330290735 Billion total entry num = 1.709792757 Billion write speed all = 27.990750659 MB/sec (with seq num) write speed all = 27.361840182 MB/sec (without seq num) 2024/11/21-00:06:11.664122 140008710874688 [compaction_job.cc:1948] [default] [JOB 22223] Generated table #88052: 926933 keys, 38143228 bytes, temperature: kWarm 2024/11/21-00:06:11.664181 140008710874688 EVENT_LOG_v1 {"time_micros": 1732118771664151, "cf_name": "default", "job": 22223, "event": "table_file_creation", "file_number": 88052, "file_size": 38143228, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 899798665, "largest_seqno": 1343540017, "table_properties": {"data_size": 31941392, "index_size": 1751808, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25954124, "raw_average_key_size": 28, "raw_value_size": 108263742, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926933, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118768, "oldest_key_time": 0, "file_creation_time": 1732118768, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 88052, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:06:11.704538 140008710874688 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088053.sst) /dev/shm/ToplingTemp/Topling-1176197-5rwN0F 2024/11/21-00:06:12.685672 140008710874688 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088053.sst): first pass time = 0.98's, 136.803'MB/sec 2024/11/21-00:06:13.865548 140008710874688 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088053.sst): old prealloc_size = 147639500, real_size = 34410736 2024/11/21-00:06:13.866137 140008710874688 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088053.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926665 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0324 GB avg-val = 116.84 avg-zval = 34.98 2024/11/21-00:06:13.866952 140008710874688 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088053.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:06:14.718809 140008710874688 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088053.sst): second pass time = 1.18's, 91.832'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.371'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3133.220'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 290.563'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33953.314'MB/sec zip my value time = 1.18's, unzip length = 0.108'GB zip my value throughput = 91.832'MB/sec zip pipeline throughput = 91.832'MB/sec entries = 926665 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.84 avg-zval = 34.98 usrkeys = 926665 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.72 avg-zval = 34.98 TagRS{ kinds = RS_Key0_TagN bytes = 144880 } TagArray{ size = 910634 bytes = 3301056 } seq expand size = 6374438 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0324 GB dict = 1.08 MB all = 0.0387 GB } UnZip/Zip{ index = 10.5146 value = 3.3400 dict = 1.00 all = 3.4671 } Zip/UnZip{ index = 0.0951 value = 0.2994 dict = 1.00 all = 0.2884 } ---------------------------- total value len = 564.318824 GB avg = 0.330 KB (by entry num) total key len = 44.600085 GB avg = 0.026 KB total ukey len = 32.998198 GB avg = 0.014 KB total ukey num = 2.331217400 Billion total entry num = 1.710719422 Billion write speed all = 27.993221094 MB/sec (with seq num) write speed all = 27.364059526 MB/sec (without seq num) 2024/11/21-00:06:14.766573 140008710874688 [compaction_job.cc:1948] [default] [JOB 22223] Generated table #88053: 926665 keys, 38711598 bytes, temperature: kWarm 2024/11/21-00:06:14.766635 140008710874688 EVENT_LOG_v1 {"time_micros": 1732118774766604, "cf_name": "default", "job": 22223, "event": "table_file_creation", "file_number": 88053, "file_size": 38711598, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1343540894, "table_properties": {"data_size": 32416288, "index_size": 1762624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25946620, "raw_average_key_size": 28, "raw_value_size": 108271134, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926665, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118771, "oldest_key_time": 0, "file_creation_time": 1732118771, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 88053, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:06:14.805641 140008710874688 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088054.sst) /dev/shm/ToplingTemp/Topling-1176197-f1TT3G 2024/11/21-00:06:15.769724 140008710874688 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088054.sst): first pass time = 0.96's, 139.222'MB/sec 2024/11/21-00:06:16.886305 140008710874688 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088054.sst): old prealloc_size = 147639500, real_size = 34496480 2024/11/21-00:06:16.886854 140008710874688 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088054.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927125 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0325 GB avg-val = 116.77 avg-zval = 35.06 2024/11/21-00:06:16.887684 140008710874688 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088054.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:06:17.725291 140008710874688 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088054.sst): second pass time = 1.12's, 97.037'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.142'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3367.643'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.13's, 308.639'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 17593.934'MB/sec zip my value time = 1.12's, unzip length = 0.108'GB zip my value throughput = 97.037'MB/sec zip pipeline throughput = 97.037'MB/sec entries = 927125 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.77 avg-zval = 35.06 usrkeys = 927125 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.77 avg-zval = 35.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927125 bytes = 3360848 } seq expand size = 6489875 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0387 GB } UnZip/Zip{ index = 10.5133 value = 3.3309 dict = 1.00 all = 3.4671 } Zip/UnZip{ index = 0.0951 value = 0.3002 dict = 1.00 all = 0.2884 } ---------------------------- total value len = 564.433572 GB avg = 0.330 KB (by entry num) total key len = 44.626045 GB avg = 0.026 KB total ukey len = 33.016741 GB avg = 0.014 KB total ukey num = 2.332144525 Billion total entry num = 1.711646547 Billion write speed all = 27.995820274 MB/sec (with seq num) write speed all = 27.366404726 MB/sec (without seq num) 2024/11/21-00:06:17.774169 140008710874688 [compaction_job.cc:1948] [default] [JOB 22223] Generated table #88054: 927125 keys, 38711538 bytes, temperature: kWarm 2024/11/21-00:06:17.774242 140008710874688 EVENT_LOG_v1 {"time_micros": 1732118777774204, "cf_name": "default", "job": 22223, "event": "table_file_creation", "file_number": 88054, "file_size": 38711538, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1053131113, "largest_seqno": 1343540856, "table_properties": {"data_size": 32500816, "index_size": 1763712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25959500, "raw_average_key_size": 28, "raw_value_size": 108258354, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927125, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118774, "oldest_key_time": 0, "file_creation_time": 1732118774, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 88054, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:06:17.809965 140008710874688 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088055.sst) /dev/shm/ToplingTemp/Topling-1176197-9pg8mI 2024/11/21-00:06:18.612920 140008710874688 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088055.sst): first pass time = 0.80's, 167.162'MB/sec 2024/11/21-00:06:19.780302 140008710874688 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088055.sst): old prealloc_size = 147639500, real_size = 35085296 2024/11/21-00:06:19.780881 140008710874688 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088055.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928653 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0331 GB avg-val = 116.53 avg-zval = 35.63 2024/11/21-00:06:19.781749 140008710874688 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088055.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:06:20.622537 140008710874688 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088055.sst): second pass time = 1.17's, 92.778'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.668'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3283.491'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 302.503'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31484.605'MB/sec zip my value time = 1.17's, unzip length = 0.108'GB zip my value throughput = 92.778'MB/sec zip pipeline throughput = 92.778'MB/sec entries = 928653 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.53 avg-zval = 35.63 usrkeys = 928653 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.53 avg-zval = 35.63 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928653 bytes = 3366384 } seq expand size = 6500571 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0331 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.5139 value = 3.2707 dict = 1.00 all = 3.4148 } Zip/UnZip{ index = 0.0951 value = 0.3057 dict = 1.00 all = 0.2928 } ---------------------------- total value len = 564.548288 GB avg = 0.330 KB (by entry num) total key len = 44.652047 GB avg = 0.026 KB total ukey len = 33.035314 GB avg = 0.014 KB total ukey num = 2.333073178 Billion total entry num = 1.712575200 Billion write speed all = 27.998559815 MB/sec (with seq num) write speed all = 27.368886634 MB/sec (without seq num) 2024/11/21-00:06:20.671688 140008710874688 [compaction_job.cc:1948] [default] [JOB 22223] Generated table #88055: 928653 keys, 39305156 bytes, temperature: kWarm 2024/11/21-00:06:20.671758 140008710874688 EVENT_LOG_v1 {"time_micros": 1732118780671727, "cf_name": "default", "job": 22223, "event": "table_file_creation", "file_number": 88055, "file_size": 39305156, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1053131138, "largest_seqno": 1343540773, "table_properties": {"data_size": 33086432, "index_size": 1766528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26002284, "raw_average_key_size": 28, "raw_value_size": 108215550, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928653, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118777, "oldest_key_time": 0, "file_creation_time": 1732118777, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 88055, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:06:20.706868 140008710874688 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088056.sst) /dev/shm/ToplingTemp/Topling-1176197-dulGsG 2024/11/21-00:06:21.502476 140008710874688 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088056.sst): first pass time = 0.80's, 168.709'MB/sec 2024/11/21-00:06:22.703433 140008710874688 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088056.sst): old prealloc_size = 147639500, real_size = 35674096 2024/11/21-00:06:22.703990 140008710874688 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088056.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930225 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1082 GB zip-val = 0.0337 GB avg-val = 116.29 avg-zval = 36.21 2024/11/21-00:06:22.704522 140008710874688 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088056.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:06:23.475651 140008710874688 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088056.sst): second pass time = 1.20's, 90.141'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.77's, 49.631'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3371.612'MB/sec (index lex order gen) rebuild zvType time = 0.64's, 0.361'MB/sec write SST data time = 0.13's, 301.641'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 19423.133'MB/sec zip my value time = 1.20's, unzip length = 0.108'GB zip my value throughput = 90.141'MB/sec zip pipeline throughput = 90.141'MB/sec entries = 930225 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.29 avg-zval = 36.21 usrkeys = 930225 avg-key = 20.00 avg-zkey = 1.89 avg-val = 119.97 avg-zval = 36.21 TagRS{ kinds = RS_Key0_TagN bytes = 145440 } TagArray{ size = 489017 bytes = 1650448 } seq expand size = 3423119 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0337 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.5819 value = 3.2114 dict = 1.00 all = 3.5020 } Zip/UnZip{ index = 0.0945 value = 0.3114 dict = 1.00 all = 0.2855 } ---------------------------- total value len = 564.659883 GB avg = 0.330 KB (by entry num) total key len = 44.678093 GB avg = 0.026 KB total ukey len = 33.053918 GB avg = 0.014 KB total ukey num = 2.334003403 Billion total entry num = 1.713505425 Billion write speed all = 28.001213988 MB/sec (with seq num) write speed all = 27.371281386 MB/sec (without seq num) 2024/11/21-00:06:23.521854 140008710874688 [compaction_job.cc:1948] [default] [JOB 22223] Generated table #88056: 930225 keys, 38325641 bytes, temperature: kWarm 2024/11/21-00:06:23.521928 140008710874688 EVENT_LOG_v1 {"time_micros": 1732118783521894, "cf_name": "default", "job": 22223, "event": "table_file_creation", "file_number": 88056, "file_size": 38325641, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1343541265, "table_properties": {"data_size": 33683232, "index_size": 1758144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26046300, "raw_average_key_size": 28, "raw_value_size": 108171450, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930225, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118780, "oldest_key_time": 0, "file_creation_time": 1732118780, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 88056, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:06:23.556353 140008710874688 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088057.sst) /dev/shm/ToplingTemp/Topling-1176197-Xi74RH 2024/11/21-00:06:24.421868 140008710874688 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088057.sst): first pass time = 0.87's, 155.081'MB/sec 2024/11/21-00:06:25.567570 140008710874688 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088057.sst): old prealloc_size = 147639500, real_size = 33170320 2024/11/21-00:06:25.568142 140008710874688 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088057.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924066 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 117.25 avg-zval = 33.74 2024/11/21-00:06:25.568979 140008710874688 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088057.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:06:26.403116 140008710874688 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088057.sst): second pass time = 1.14's, 94.643'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.818'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3296.766'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.328'MB/sec write SST data time = 0.13's, 288.791'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 19969.700'MB/sec zip my value time = 1.14's, unzip length = 0.108'GB zip my value throughput = 94.643'MB/sec zip pipeline throughput = 94.643'MB/sec entries = 924066 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.25 avg-zval = 33.74 usrkeys = 924066 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.08 avg-zval = 33.74 TagRS{ kinds = RS_Key0_TagN bytes = 144480 } TagArray{ size = 902485 bytes = 3271520 } seq expand size = 6317395 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5160 value = 3.4746 dict = 1.00 all = 3.5842 } Zip/UnZip{ index = 0.0951 value = 0.2878 dict = 1.00 all = 0.2790 } ---------------------------- total value len = 564.774544 GB avg = 0.329 KB (by entry num) total key len = 44.703967 GB avg = 0.026 KB total ukey len = 33.072400 GB avg = 0.014 KB total ukey num = 2.334927469 Billion total entry num = 1.714429491 Billion write speed all = 28.003904784 MB/sec (with seq num) write speed all = 27.373717248 MB/sec (without seq num) 2024/11/21-00:06:26.454331 140008710874688 [compaction_job.cc:1948] [default] [JOB 22223] Generated table #88057: 924066 keys, 37447078 bytes, temperature: kWarm 2024/11/21-00:06:26.454402 140008710874688 EVENT_LOG_v1 {"time_micros": 1732118786454370, "cf_name": "default", "job": 22223, "event": "table_file_creation", "file_number": 88057, "file_size": 37447078, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1331717794, "table_properties": {"data_size": 31181696, "index_size": 1757440, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25873848, "raw_average_key_size": 28, "raw_value_size": 108344016, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924066, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118783, "oldest_key_time": 0, "file_creation_time": 1732118783, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 88057, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:06:26.489682 140008710874688 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088058.sst) /dev/shm/ToplingTemp/Topling-1176197-qYSRsJ 2024/11/21-00:06:27.497998 140008710874688 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088058.sst): first pass time = 1.01's, 133.117'MB/sec 2024/11/21-00:06:28.689922 140008710874688 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088058.sst): old prealloc_size = 147639500, real_size = 35446128 2024/11/21-00:06:28.690499 140008710874688 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088058.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925707 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0335 GB avg-val = 116.99 avg-zval = 36.14 2024/11/21-00:06:28.691281 140008710874688 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088058.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:06:29.519789 140008710874688 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088058.sst): second pass time = 1.19's, 90.923'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.648'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3238.832'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.12's, 322.868'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33111.976'MB/sec zip my value time = 1.19's, unzip length = 0.108'GB zip my value throughput = 90.923'MB/sec zip pipeline throughput = 90.923'MB/sec entries = 925707 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.99 avg-zval = 36.14 usrkeys = 925707 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.99 avg-zval = 36.14 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925707 bytes = 3239984 } seq expand size = 6479949 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0335 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.5133 value = 3.2373 dict = 1.00 all = 3.3944 } Zip/UnZip{ index = 0.0951 value = 0.3089 dict = 1.00 all = 0.2946 } ---------------------------- total value len = 564.889322 GB avg = 0.329 KB (by entry num) total key len = 44.729887 GB avg = 0.026 KB total ukey len = 33.090914 GB avg = 0.014 KB total ukey num = 2.335853176 Billion total entry num = 1.715355198 Billion write speed all = 28.006358879 MB/sec (with seq num) write speed all = 27.375921353 MB/sec (without seq num) 2024/11/21-00:06:29.568663 140008710874688 [compaction_job.cc:1948] [default] [JOB 22223] Generated table #88058: 925707 keys, 39541006 bytes, temperature: kWarm 2024/11/21-00:06:29.568732 140008710874688 EVENT_LOG_v1 {"time_micros": 1732118789568701, "cf_name": "default", "job": 22223, "event": "table_file_creation", "file_number": 88058, "file_size": 39541006, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1181841034, "largest_seqno": 1331717027, "table_properties": {"data_size": 33453472, "index_size": 1761024, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25919796, "raw_average_key_size": 28, "raw_value_size": 108298030, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925707, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118786, "oldest_key_time": 0, "file_creation_time": 1732118786, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 88058, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:06:29.604818 140008710874688 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088059.sst) /dev/shm/ToplingTemp/Topling-1176197-Kz2cfJ 2024/11/21-00:06:30.626457 140008710874688 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088059.sst): first pass time = 1.02's, 131.380'MB/sec 2024/11/21-00:06:31.797354 140008710874688 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088059.sst): old prealloc_size = 147639500, real_size = 35191952 2024/11/21-00:06:31.797967 140008710874688 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088059.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924912 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0332 GB avg-val = 117.11 avg-zval = 35.87 2024/11/21-00:06:31.798744 140008710874688 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088059.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:06:32.637182 140008710874688 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088059.sst): second pass time = 1.17's, 92.574'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.641'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3031.420'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 295.177'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 22941.179'MB/sec zip my value time = 1.17's, unzip length = 0.108'GB zip my value throughput = 92.574'MB/sec zip pipeline throughput = 92.574'MB/sec entries = 924912 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.11 avg-zval = 35.87 usrkeys = 924912 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.11 avg-zval = 35.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924912 bytes = 3121600 } seq expand size = 6474384 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0392 GB } UnZip/Zip{ index = 10.3760 value = 3.2648 dict = 1.00 all = 3.4266 } Zip/UnZip{ index = 0.0964 value = 0.3063 dict = 1.00 all = 0.2918 } ---------------------------- total value len = 565.004117 GB avg = 0.329 KB (by entry num) total key len = 44.755785 GB avg = 0.026 KB total ukey len = 33.109412 GB avg = 0.014 KB total ukey num = 2.336778088 Billion total entry num = 1.716280110 Billion write speed all = 28.008811090 MB/sec (with seq num) write speed all = 27.378123959 MB/sec (without seq num) 2024/11/21-00:06:32.685259 140008710874688 [compaction_job.cc:1948] [default] [JOB 22223] Generated table #88059: 924912 keys, 39169334 bytes, temperature: kWarm 2024/11/21-00:06:32.685321 140008710874688 EVENT_LOG_v1 {"time_micros": 1732118792685290, "cf_name": "default", "job": 22223, "event": "table_file_creation", "file_number": 88059, "file_size": 39169334, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1209463308, "largest_seqno": 1331716841, "table_properties": {"data_size": 33177728, "index_size": 1782784, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25897536, "raw_average_key_size": 28, "raw_value_size": 108320296, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924912, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118789, "oldest_key_time": 0, "file_creation_time": 1732118789, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 88059, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:06:32.719385 140008710874688 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088060.sst) /dev/shm/ToplingTemp/Topling-1176197-d4cgBI 2024/11/21-00:06:33.573631 140008710874688 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088060.sst): first pass time = 0.85's, 157.124'MB/sec 2024/11/21-00:06:34.710111 140008710874688 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088060.sst): old prealloc_size = 147639500, real_size = 37964880 2024/11/21-00:06:34.710684 140008710874688 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088060.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927718 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0360 GB avg-val = 116.68 avg-zval = 38.77 2024/11/21-00:06:34.711507 140008710874688 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088060.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:06:35.510751 140008710874688 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088060.sst): second pass time = 1.14's, 95.312'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.80's, 52.691'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3210.984'MB/sec (index lex order gen) rebuild zvType time = 0.67's, 0.348'MB/sec write SST data time = 0.13's, 317.994'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33090.085'MB/sec zip my value time = 1.14's, unzip length = 0.108'GB zip my value throughput = 95.312'MB/sec zip pipeline throughput = 95.312'MB/sec entries = 927718 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.68 avg-zval = 38.77 usrkeys = 927718 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.68 avg-zval = 38.77 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927718 bytes = 3362992 } seq expand size = 6494026 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0360 GB dict = 1.08 MB all = 0.0422 GB } UnZip/Zip{ index = 10.5113 value = 3.0094 dict = 1.00 all = 3.1816 } Zip/UnZip{ index = 0.0951 value = 0.3323 dict = 1.00 all = 0.3143 } ---------------------------- total value len = 565.118853 GB avg = 0.329 KB (by entry num) total key len = 44.781761 GB avg = 0.026 KB total ukey len = 33.127966 GB avg = 0.014 KB total ukey num = 2.337705806 Billion total entry num = 1.717207828 Billion write speed all = 28.011577177 MB/sec (with seq num) write speed all = 27.380632416 MB/sec (without seq num) 2024/11/21-00:06:35.563703 140008710874688 [compaction_job.cc:1948] [default] [JOB 22223] Generated table #88060: 927718 keys, 42185716 bytes, temperature: kWarm 2024/11/21-00:06:35.563766 140008710874688 EVENT_LOG_v1 {"time_micros": 1732118795563736, "cf_name": "default", "job": 22223, "event": "table_file_creation", "file_number": 88060, "file_size": 42185716, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 951052351, "largest_seqno": 1331708274, "table_properties": {"data_size": 35967552, "index_size": 1765184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25976104, "raw_average_key_size": 28, "raw_value_size": 108241688, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927718, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118792, "oldest_key_time": 0, "file_creation_time": 1732118792, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 88060, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:06:35.597992 140008710874688 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088061.sst) /dev/shm/ToplingTemp/Topling-1176197-TdlkiH 2024/11/21-00:06:36.509686 140008710874688 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088061.sst): first pass time = 0.91's, 147.224'MB/sec 2024/11/21-00:06:37.665083 140008710874688 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088061.sst): old prealloc_size = 147639500, real_size = 34301744 2024/11/21-00:06:37.665672 140008710874688 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088061.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923573 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0323 GB avg-val = 117.32 avg-zval = 34.98 2024/11/21-00:06:37.666481 140008710874688 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088061.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:06:38.513696 140008710874688 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088061.sst): second pass time = 1.15's, 93.856'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.408'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3060.240'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.323'MB/sec write SST data time = 0.13's, 287.695'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 30332.754'MB/sec zip my value time = 1.15's, unzip length = 0.108'GB zip my value throughput = 93.856'MB/sec zip pipeline throughput = 93.856'MB/sec entries = 923573 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.32 avg-zval = 34.98 usrkeys = 923573 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.31 avg-zval = 34.98 TagRS{ kinds = RS_Key0_TagN bytes = 144400 } TagArray{ size = 921956 bytes = 3226864 } seq expand size = 6453692 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0323 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.4458 value = 3.3545 dict = 1.00 all = 3.4831 } Zip/UnZip{ index = 0.0957 value = 0.2981 dict = 1.00 all = 0.2871 } ---------------------------- total value len = 565.233664 GB avg = 0.329 KB (by entry num) total key len = 44.807621 GB avg = 0.026 KB total ukey len = 33.146438 GB avg = 0.014 KB total ukey num = 2.338629379 Billion total entry num = 1.718131401 Billion write speed all = 28.014174261 MB/sec (with seq num) write speed all = 27.382977210 MB/sec (without seq num) 2024/11/21-00:06:38.561904 140008710874688 [compaction_job.cc:1948] [default] [JOB 22223] Generated table #88061: 923573 keys, 38534304 bytes, temperature: kWarm 2024/11/21-00:06:38.561962 140008710874688 EVENT_LOG_v1 {"time_micros": 1732118798561932, "cf_name": "default", "job": 22223, "event": "table_file_creation", "file_number": 88061, "file_size": 38534304, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1343541474, "table_properties": {"data_size": 32302368, "index_size": 1768320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25860044, "raw_average_key_size": 28, "raw_value_size": 108357806, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923573, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118795, "oldest_key_time": 0, "file_creation_time": 1732118795, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 88061, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:06:38.596991 140008710874688 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088062.sst) /dev/shm/ToplingTemp/Topling-1176197-XkRUOG 2024/11/21-00:06:38.707763 140008710874688 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088062.sst): first pass time = 0.11's, 258.987'MB/sec 2024/11/21-00:06:38.943479 140008710874688 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088062.sst): old prealloc_size = 147639500, real_size = 8074224 2024/11/21-00:06:38.943653 140008710874688 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088062.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 197946 prefix = 4 raw-key = 0.0032 GB zip-key = 0.0004 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0231 GB zip-val = 0.0076 GB avg-val = 116.89 avg-zval = 38.62 2024/11/21-00:06:38.943849 140008710874688 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088062.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0004 GB, waited 0.000 sec, Key+Value = 0.029 GB 2024/11/21-00:06:39.119648 140008710874688 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00/088062.sst): second pass time = 0.24's, 98.442'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 50.589'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2481.592'MB/sec (index lex order gen) rebuild zvType time = 0.15's, 0.335'MB/sec write SST data time = 0.03's, 316.203'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.241'MB, throughput = 9609.919'MB/sec zip my value time = 0.24's, unzip length = 0.023'GB zip my value throughput = 98.442'MB/sec zip pipeline throughput = 98.442'MB/sec entries = 197946 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.89 avg-zval = 38.62 usrkeys = 197946 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.89 avg-zval = 38.62 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 197946 bytes = 643344 } seq expand size = 1385622 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0040 GB value = 0.0231 GB dict = 0.24 MB all = 0.0287 GB } __ZipSize{ index = 0.0004 GB value = 0.0076 GB dict = 0.24 MB all = 0.0089 GB } UnZip/Zip{ index = 10.4138 value = 3.0269 dict = 1.00 all = 3.2185 } Zip/UnZip{ index = 0.0960 value = 0.3304 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 565.258188 GB avg = 0.329 KB (by entry num) total key len = 44.813163 GB avg = 0.026 KB total ukey len = 33.150397 GB avg = 0.014 KB total ukey num = 2.338827325 Billion total entry num = 1.718329347 Billion write speed all = 28.014775419 MB/sec (with seq num) write speed all = 27.383523214 MB/sec (without seq num) 2024/11/21-00:06:39.136916 140008710874688 [compaction_job.cc:1948] [default] [JOB 22223] Generated table #88062: 197946 keys, 8911141 bytes, temperature: kWarm 2024/11/21-00:06:39.136969 140008710874688 EVENT_LOG_v1 {"time_micros": 1732118799136941, "cf_name": "default", "job": 22223, "event": "table_file_creation", "file_number": 88062, "file_size": 8911141, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1294941875, "largest_seqno": 1343541486, "table_properties": {"data_size": 7644384, "index_size": 380160, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 5542488, "raw_average_key_size": 28, "raw_value_size": 23138452, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 197946, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118798, "oldest_key_time": 0, "file_creation_time": 1732118798, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 88062, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:06:39.149299 140008710874688 [compaction_job.cc:703] job-22223: subcompact[0], size: 0.756406 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22223/att-00 [088043,088044,088045,088046,088047,088048,088049,088050,088051,088052,088053,088054,088055,088056,088057,088058,088059,088060,088061,088062] 2024/11/21-00:06:39.266691 140008710874688 (Original Log Time 2024/11/21-00:05:41.496486) EVENT_LOG_v1 {"time_micros": 1732118741496462, "job": 22223, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [87982], "files_L3": [88009, 88010, 87930, 87931, 87932, 87933, 87934, 87935, 87936, 87937, 87938, 87939, 87942, 87943, 87944, 87945, 87946, 87947, 87948, 88029], "score": 0.993559, "input_data_size": 780936621, "oldest_snapshot_seqno": 1343995949} 2024/11/21-00:06:39.266694 140008710874688 (Original Log Time 2024/11/21-00:06:39.265104) [compaction_job.cc:2022] [default] [JOB 22223] Compacted 1@2 + 20@3 files to L3 => 756406263 bytes 2024/11/21-00:06:39.266696 140008710874688 (Original Log Time 2024/11/21-00:06:39.266621) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 20 0 0 0] max score 0.03, MB/sec: 13.5 rd, 13.1 wr, level 3, files in(1, 20) out(20 +0 blob) MB in(27.1, 717.7 +0.0 blob) out(721.4 +0.0 blob), read-write-amplify(54.1) write-amplify(26.6) OK, records in: 17915241, records dropped: 102492 output_compression: Snappy 2024/11/21-00:06:39.266701 140008710874688 (Original Log Time 2024/11/21-00:06:39.266670) EVENT_LOG_v1 {"time_micros": 1732118799266644, "cf": "default", "job": 22223, "event": "compaction_finished", "compaction_time_micros": 57652818, "compaction_time_cpu_micros": 25519851, "output_level": 3, "num_output_files": 20, "total_output_size": 756406263, "num_input_records": 17915241, "num_output_records": 17812749, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 517243533, "file_range_sync_nanos": 0, "file_fsync_nanos": 17055, "file_prepare_write_nanos": 53497, "lsm_state": [0, 0, 0, 20, 0, 0, 0]} 2024/11/21-00:06:39.385690 140008710874688 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732096383,"starts":1732118741,"executesMs":57947,"compactionJobId":22223,"attempt":0,"compactionInputRawBytes":2593373278,"compactionInputZipBytes":780936621,"compactionOutputRawBytes":0,"compactionOutputZipBytes":756406263,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-dad211a0d5f9d9d8c3206644539b1a2e-eb4a1da89ba05ba6-00"}}, response = 2024/11/21-00:06:39.385745 140008710874688 INFO dcompact_worker.cpp:1464: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-22223/att-00: olev 3, work 57.947 s, result 7.624 ms, install 1.612 ms, input{raw 2.415 GiB zip 744.759 MiB}