2024/11/20-18:27:58.995632 140008517842496 DEBG dcompact_worker.cpp:2219: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00: fork to child time = 0.009981 sec 2024/11/20-18:27:58.996281 140008517842496 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00 2024/11/20-18:27:59.003506 140008517842496 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00 2024/11/20-18:27:59.005449 140008517842496 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/20-18:27:59.005739 140008517842496 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/20-18:27:59.052238 140008517842496 [version_set.cc:6211] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-04908/att-00/MANIFEST-049091 2024/11/20-18:27:59.070496 140008517842496 [version_set.cc:6254] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-04908/att-00/MANIFEST-049091 succeeded,manifest_file_number is 49091, next_file_number is 56022, last_sequence is 945621975, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 55084 2024/11/20-18:27:59.070504 140008517842496 [version_set.cc:6269] Column family [default] (ID 0), log number is 0 2024/11/20-18:27:59.070648 140008517842496 [version_set.cc:5724] Creating manifest 56022 2024/11/20-18:27:59.105918 140008517842496 DEBG dcompact_worker.cpp:1225: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/20-18:27:59.129165 140008517842496 [compaction_job.cc:2437] [default] [JOB 4908] Compacting 1@2 + 52@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/20-18:27:59.129201 140008517842496 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 4640 Base level 2, inputs: [56017(31MB)], [55958(39MB) 55959(39MB) 55960(39MB) 55961(39MB) 55963(39MB) 55964(39MB) 55965(39MB) 55966(39MB) 55967(39MB) 55968(39MB) 55970(39MB) 55971(39MB) 55972(39MB) 55973(39MB) 55974(39MB) 55975(39MB) 55976(39MB) 55977(39MB) 55978(39MB) 55979(39MB) 55980(39MB) 55981(39MB) 55982(39MB) 55983(39MB) 55984(39MB) 55985(39MB) 55986(39MB) 55987(39MB) 55988(39MB) 55989(39MB) 55990(39MB) 55991(39MB) 55992(39MB) 55993(39MB) 55995(39MB) 55996(39MB) 55997(39MB) 55998(39MB) 55999(39MB) 56000(39MB) 56001(39MB) 56002(39MB) 56003(39MB) 56004(39MB) 56005(14MB) 55692(10150KB) 55686(10118KB) 55664(10175KB) 55668(10126KB) 55674(10119KB) 55681(10176KB) 55675(8124KB)] 2024/11/20-18:27:59.137651 140008517842496 [top_zip_table.cc:59] core git_version_hash_info_is:commit 81964c21ebbcc178d8e99dae0fbf61bded2729cc Author: rockeet Date: Mon Nov 11 21:28:46 2024 +0800 Makefile: fix gen git-version-*.cpp GIT_PATH_ARG = :!src/terark/fsa :!tools/fsa :!src/terark/zbs :!tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-18:27:59.137677 140008517842496 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit 5dfecb7d107196f044ff5bf78dc39d2e725b9d9b Author: leipeng Date: Thu Nov 7 15:22:45 2024 +0800 Makefile: fix for x86_64 CPU older than haswell GIT_PATH_ARG = src/terark/fsa tools/fsa g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-18:27:59.137697 140008517842496 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit d44dd5ff72707128fe3c916077b43d2093c01f7c Author: leipeng Date: Sun Oct 27 23:38:26 2024 +0800 Fix for msvc, when toplingdb on msvc GIT_PATH_ARG = src/terark/zbs tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-18:27:59.137713 140008517842496 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit 79614a7e7dc27362fc5151b0797eb7d83c674c7b Author: rockeet Date: Sun Nov 10 15:02:21 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-18:27:59.137717 140008517842496 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit c46b65a52d9c74aa8b87aeb8438620ff2dbc669c Author: rockeet Date: Sun Nov 10 15:02:42 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-18:27:59.913647 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056023.sst) /dev/shm/ToplingTemp/Topling-806427-0SB7Qo 2024/11/20-18:28:00.081757 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056023.sst): first pass time = 0.17's, 798.597'MB/sec 2024/11/20-18:28:00.649409 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056023.sst): old prealloc_size = 147639500, real_size = 40351648 2024/11/20-18:28:00.649534 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056023.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.00 avg-zval = 309.82 2024/11/20-18:28:00.649602 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056023.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:00.818554 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056023.sst): second pass time = 0.57's, 232.177'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 246.652'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2121.109'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.264'MB/sec write SST data time = 0.05's, 894.667'MB/sec dict compress time = 4481993598.45's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45430.159'MB/sec zip my value time = 0.57's, unzip length = 0.132'GB zip my value throughput = 232.177'MB/sec zip pipeline throughput = 252.976'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.82 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.82 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 106 bytes = 352 } seq expand size = 742 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0876 value = 3.2825 dict = 1.00 all = 3.2172 } Zip/UnZip{ index = 0.1411 value = 0.3046 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 0.131630 GB avg = 1.017 KB (by entry num) total key len = 0.002589 GB avg = 0.020 KB total ukey len = 0.001553 GB avg = 0.012 KB total ukey num = 0.000129429 Billion total entry num = 0.000129429 Billion write speed all = 0.003854751 MB/sec (with seq num) write speed all = 0.003825013 MB/sec (without seq num) 2024/11/20-18:28:00.869102 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56023: 129429 keys, 41719410 bytes, temperature: kWarm 2024/11/20-18:28:00.869190 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098480869148, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56023, "file_size": 41719410, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 939521582, "table_properties": {"data_size": 40099984, "index_size": 219136, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629163, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098479, "oldest_key_time": 0, "file_creation_time": 1732098479, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56023, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:00.909104 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056024.sst) /dev/shm/ToplingTemp/Topling-806427-g7bECs 2024/11/20-18:28:01.434818 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056024.sst): first pass time = 0.53's, 255.327'MB/sec 2024/11/20-18:28:02.118380 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056024.sst): old prealloc_size = 147639500, real_size = 40353456 2024/11/20-18:28:02.118521 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056024.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129426 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.03 avg-zval = 309.85 2024/11/20-18:28:02.118562 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056024.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:02.283781 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056024.sst): second pass time = 0.68's, 192.794'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 252.122'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1947.247'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.271'MB/sec write SST data time = 0.05's, 907.258'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 40427.867'MB/sec zip my value time = 0.68's, unzip length = 0.132'GB zip my value throughput = 192.794'MB/sec zip pipeline throughput = 239.512'MB/sec entries = 129426 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.03 avg-zval = 309.85 usrkeys = 129426 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.03 avg-zval = 309.85 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0978 value = 3.2824 dict = 1.00 all = 3.2186 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 0.263260 GB avg = 1.017 KB (by entry num) total key len = 0.005177 GB avg = 0.020 KB total ukey len = 0.003106 GB avg = 0.012 KB total ukey num = 0.000258855 Billion total entry num = 0.000258855 Billion write speed all = 0.007709172 MB/sec (with seq num) write speed all = 0.007649700 MB/sec (without seq num) 2024/11/20-18:28:02.336602 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56024: 129426 keys, 41700705 bytes, temperature: kWarm 2024/11/20-18:28:02.336661 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098482336631, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56024, "file_size": 41700705, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40102112, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588520, "raw_average_key_size": 20, "raw_value_size": 131629793, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129426, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098480, "oldest_key_time": 0, "file_creation_time": 1732098480, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56024, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:02.374648 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056025.sst) /dev/shm/ToplingTemp/Topling-806427-mS2Xwq 2024/11/20-18:28:02.921585 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056025.sst): first pass time = 0.55's, 245.413'MB/sec 2024/11/20-18:28:03.441998 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056025.sst): old prealloc_size = 147639500, real_size = 40352576 2024/11/20-18:28:03.442121 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056025.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.99 avg-zval = 309.83 2024/11/20-18:28:03.442170 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056025.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:03.614990 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056025.sst): second pass time = 0.52's, 253.249'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 241.198'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2170.008'MB/sec (index lex order gen) rebuild zvType time = 0.13's, 0.247'MB/sec write SST data time = 0.04's, 996.315'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 51092.722'MB/sec zip my value time = 0.52's, unzip length = 0.132'GB zip my value throughput = 253.249'MB/sec zip pipeline throughput = 253.249'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.83 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.83 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 60 bytes = 192 } seq expand size = 420 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0959 value = 3.2824 dict = 1.00 all = 3.2171 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 0.394890 GB avg = 1.017 KB (by entry num) total key len = 0.007766 GB avg = 0.020 KB total ukey len = 0.004659 GB avg = 0.012 KB total ukey num = 0.000388285 Billion total entry num = 0.000388285 Billion write speed all = 0.011563317 MB/sec (with seq num) write speed all = 0.011474112 MB/sec (without seq num) 2024/11/20-18:28:03.665998 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56025: 129430 keys, 41720587 bytes, temperature: kWarm 2024/11/20-18:28:03.666066 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098483666024, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56025, "file_size": 41720587, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 939480253, "table_properties": {"data_size": 40101168, "index_size": 218880, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629425, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098482, "oldest_key_time": 0, "file_creation_time": 1732098482, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56025, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:03.702914 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056026.sst) /dev/shm/ToplingTemp/Topling-806427-Iy1FIp 2024/11/20-18:28:04.253388 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056026.sst): first pass time = 0.55's, 243.836'MB/sec 2024/11/20-18:28:04.878294 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056026.sst): old prealloc_size = 147639500, real_size = 40351728 2024/11/20-18:28:04.878416 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056026.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.01 avg-zval = 309.83 2024/11/20-18:28:04.878446 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056026.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:05.041390 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056026.sst): second pass time = 0.62's, 210.869'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 255.691'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2399.482'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.05's, 922.116'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45336.854'MB/sec zip my value time = 0.62's, unzip length = 0.132'GB zip my value throughput = 210.869'MB/sec zip pipeline throughput = 210.869'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.83 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.83 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2825 dict = 1.00 all = 3.2187 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 0.526519 GB avg = 1.017 KB (by entry num) total key len = 0.010354 GB avg = 0.020 KB total ukey len = 0.006213 GB avg = 0.012 KB total ukey num = 0.000517713 Billion total entry num = 0.000517713 Billion write speed all = 0.015417107 MB/sec (with seq num) write speed all = 0.015298171 MB/sec (without seq num) 2024/11/20-18:28:05.093630 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56026: 129428 keys, 41699133 bytes, temperature: kWarm 2024/11/20-18:28:05.093694 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098485093663, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56026, "file_size": 41699133, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40100384, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629227, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098483, "oldest_key_time": 0, "file_creation_time": 1732098483, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56026, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:05.132377 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056027.sst) /dev/shm/ToplingTemp/Topling-806427-MdDNxo 2024/11/20-18:28:05.794010 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056027.sst): first pass time = 0.66's, 202.869'MB/sec 2024/11/20-18:28:06.287328 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056027.sst): old prealloc_size = 147639500, real_size = 40351824 2024/11/20-18:28:06.287476 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056027.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.00 avg-zval = 309.83 2024/11/20-18:28:06.287515 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056027.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:06.452240 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056027.sst): second pass time = 0.49's, 267.274'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 252.864'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1841.359'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.278'MB/sec write SST data time = 0.05's, 863.628'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 28490.420'MB/sec zip my value time = 0.49's, unzip length = 0.132'GB zip my value throughput = 267.274'MB/sec zip pipeline throughput = 267.274'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.83 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.83 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2825 dict = 1.00 all = 3.2187 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 0.658148 GB avg = 1.017 KB (by entry num) total key len = 0.012943 GB avg = 0.020 KB total ukey len = 0.007766 GB avg = 0.012 KB total ukey num = 0.000647142 Billion total entry num = 0.000647142 Billion write speed all = 0.019270604 MB/sec (with seq num) write speed all = 0.019121941 MB/sec (without seq num) 2024/11/20-18:28:06.501990 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56027: 129429 keys, 41699533 bytes, temperature: kWarm 2024/11/20-18:28:06.502061 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098486502021, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56027, "file_size": 41699533, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40100480, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629724, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098485, "oldest_key_time": 0, "file_creation_time": 1732098485, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56027, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:06.539210 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056028.sst) /dev/shm/ToplingTemp/Topling-806427-Onb01r 2024/11/20-18:28:07.140614 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056028.sst): first pass time = 0.60's, 223.190'MB/sec 2024/11/20-18:28:07.683074 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056028.sst): old prealloc_size = 147639500, real_size = 40351552 2024/11/20-18:28:07.683194 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056028.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.01 avg-zval = 309.83 2024/11/20-18:28:07.683231 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056028.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:07.843185 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056028.sst): second pass time = 0.54's, 243.022'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 260.431'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2211.178'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.282'MB/sec write SST data time = 0.05's, 924.296'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 24286.610'MB/sec zip my value time = 0.54's, unzip length = 0.132'GB zip my value throughput = 243.022'MB/sec zip pipeline throughput = 243.022'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.83 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.83 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2825 dict = 1.00 all = 3.2187 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 0.789778 GB avg = 1.017 KB (by entry num) total key len = 0.015531 GB avg = 0.020 KB total ukey len = 0.009319 GB avg = 0.012 KB total ukey num = 0.000776570 Billion total entry num = 0.000776570 Billion write speed all = 0.023123801 MB/sec (with seq num) write speed all = 0.022945412 MB/sec (without seq num) 2024/11/20-18:28:07.892875 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56028: 129428 keys, 41699220 bytes, temperature: kWarm 2024/11/20-18:28:07.892937 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098487892907, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56028, "file_size": 41699220, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40100208, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629693, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098486, "oldest_key_time": 0, "file_creation_time": 1732098486, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56028, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:07.927211 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056029.sst) /dev/shm/ToplingTemp/Topling-806427-Bt9u1r 2024/11/20-18:28:08.621960 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056029.sst): first pass time = 0.69's, 193.198'MB/sec 2024/11/20-18:28:09.100707 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056029.sst): old prealloc_size = 147639500, real_size = 40349264 2024/11/20-18:28:09.100836 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056029.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129432 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.98 avg-zval = 309.80 2024/11/20-18:28:09.100871 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056029.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:09.257808 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056029.sst): second pass time = 0.48's, 275.393'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 265.443'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2087.083'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.284'MB/sec write SST data time = 0.04's, 968.868'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 42642.795'MB/sec zip my value time = 0.48's, unzip length = 0.132'GB zip my value throughput = 275.393'MB/sec zip pipeline throughput = 275.393'MB/sec entries = 129432 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.98 avg-zval = 309.80 usrkeys = 129432 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.98 avg-zval = 309.80 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0981 value = 3.2827 dict = 1.00 all = 3.2189 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 0.921407 GB avg = 1.017 KB (by entry num) total key len = 0.018120 GB avg = 0.020 KB total ukey len = 0.010872 GB avg = 0.012 KB total ukey num = 0.000906002 Billion total entry num = 0.000906002 Billion write speed all = 0.026976662 MB/sec (with seq num) write speed all = 0.026768550 MB/sec (without seq num) 2024/11/20-18:28:09.307483 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56029: 129432 keys, 41696452 bytes, temperature: kWarm 2024/11/20-18:28:09.307544 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098489307515, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56029, "file_size": 41696452, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40097920, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588640, "raw_average_key_size": 20, "raw_value_size": 131629305, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098487, "oldest_key_time": 0, "file_creation_time": 1732098487, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56029, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:09.344518 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056030.sst) /dev/shm/ToplingTemp/Topling-806427-L76hxq 2024/11/20-18:28:10.037891 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056030.sst): first pass time = 0.69's, 193.581'MB/sec 2024/11/20-18:28:10.599477 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056030.sst): old prealloc_size = 147639500, real_size = 40350416 2024/11/20-18:28:10.599593 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056030.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.01 avg-zval = 309.82 2024/11/20-18:28:10.599627 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056030.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:10.755659 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056030.sst): second pass time = 0.56's, 234.717'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 267.003'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2261.197'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.284'MB/sec write SST data time = 0.04's, 986.181'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 38195.471'MB/sec zip my value time = 0.56's, unzip length = 0.132'GB zip my value throughput = 234.717'MB/sec zip pipeline throughput = 234.717'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.82 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.82 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2826 dict = 1.00 all = 3.2188 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 1.053037 GB avg = 1.017 KB (by entry num) total key len = 0.020709 GB avg = 0.020 KB total ukey len = 0.012425 GB avg = 0.012 KB total ukey num = 0.001035430 Billion total entry num = 0.001035430 Billion write speed all = 0.030829140 MB/sec (with seq num) write speed all = 0.030591308 MB/sec (without seq num) 2024/11/20-18:28:10.805739 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56030: 129428 keys, 41697899 bytes, temperature: kWarm 2024/11/20-18:28:10.805792 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098490805764, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56030, "file_size": 41697899, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40099072, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629476, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098489, "oldest_key_time": 0, "file_creation_time": 1732098489, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56030, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:10.842604 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056031.sst) /dev/shm/ToplingTemp/Topling-806427-x0hK2o 2024/11/20-18:28:11.391197 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056031.sst): first pass time = 0.55's, 244.673'MB/sec 2024/11/20-18:28:11.945282 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056031.sst): old prealloc_size = 147639500, real_size = 40353088 2024/11/20-18:28:11.945417 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056031.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.02 avg-zval = 309.84 2024/11/20-18:28:11.945456 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056031.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:12.109339 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056031.sst): second pass time = 0.55's, 237.921'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 254.196'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2173.208'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.271'MB/sec write SST data time = 0.04's, 932.573'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 28866.495'MB/sec zip my value time = 0.55's, unzip length = 0.132'GB zip my value throughput = 237.921'MB/sec zip pipeline throughput = 237.921'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.84 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.84 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2824 dict = 1.00 all = 3.2186 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 1.184666 GB avg = 1.017 KB (by entry num) total key len = 0.023297 GB avg = 0.020 KB total ukey len = 0.013978 GB avg = 0.012 KB total ukey num = 0.001164857 Billion total entry num = 0.001164857 Billion write speed all = 0.034681423 MB/sec (with seq num) write speed all = 0.034413873 MB/sec (without seq num) 2024/11/20-18:28:12.161025 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56031: 129427 keys, 41700389 bytes, temperature: kWarm 2024/11/20-18:28:12.161092 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098492161060, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56031, "file_size": 41700389, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40101744, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629253, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098490, "oldest_key_time": 0, "file_creation_time": 1732098490, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56031, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:12.201763 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056032.sst) /dev/shm/ToplingTemp/Topling-806427-Xgp4sq 2024/11/20-18:28:12.734778 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056032.sst): first pass time = 0.53's, 251.824'MB/sec 2024/11/20-18:28:13.286313 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056032.sst): old prealloc_size = 147639500, real_size = 40347328 2024/11/20-18:28:13.286443 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056032.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.99 avg-zval = 309.79 2024/11/20-18:28:13.286479 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056032.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:13.485728 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056032.sst): second pass time = 0.55's, 238.986'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.20's, 209.097'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2025.755'MB/sec (index lex order gen) rebuild zvType time = 0.15's, 0.217'MB/sec write SST data time = 0.05's, 829.810'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 35566.311'MB/sec zip my value time = 0.55's, unzip length = 0.132'GB zip my value throughput = 238.986'MB/sec zip pipeline throughput = 238.986'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.79 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.79 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0981 value = 3.2829 dict = 1.00 all = 3.2191 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3106 } ---------------------------- total value len = 1.316296 GB avg = 1.017 KB (by entry num) total key len = 0.025886 GB avg = 0.020 KB total ukey len = 0.015531 GB avg = 0.012 KB total ukey num = 0.001294288 Billion total entry num = 0.001294288 Billion write speed all = 0.038533391 MB/sec (with seq num) write speed all = 0.038236124 MB/sec (without seq num) 2024/11/20-18:28:13.554914 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56032: 129431 keys, 41694817 bytes, temperature: kWarm 2024/11/20-18:28:13.554964 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098493554937, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56032, "file_size": 41694817, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40095984, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629511, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098492, "oldest_key_time": 0, "file_creation_time": 1732098492, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56032, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:13.595112 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056033.sst) /dev/shm/ToplingTemp/Topling-806427-PNJDYo 2024/11/20-18:28:14.159270 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056033.sst): first pass time = 0.56's, 237.927'MB/sec 2024/11/20-18:28:14.703433 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056033.sst): old prealloc_size = 147639500, real_size = 40347248 2024/11/20-18:28:14.703559 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056033.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129435 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.95 avg-zval = 309.78 2024/11/20-18:28:14.703592 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056033.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:14.870092 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056033.sst): second pass time = 0.54's, 242.226'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 250.241'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2168.234'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.272'MB/sec write SST data time = 0.05's, 874.205'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 49706.676'MB/sec zip my value time = 0.54's, unzip length = 0.132'GB zip my value throughput = 242.226'MB/sec zip pipeline throughput = 249.731'MB/sec entries = 129435 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.95 avg-zval = 309.78 usrkeys = 129435 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.95 avg-zval = 309.78 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0983 value = 3.2829 dict = 1.00 all = 3.2191 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 1.447925 GB avg = 1.017 KB (by entry num) total key len = 0.028474 GB avg = 0.020 KB total ukey len = 0.017085 GB avg = 0.012 KB total ukey num = 0.001423723 Billion total entry num = 0.001423723 Billion write speed all = 0.042385035 MB/sec (with seq num) write speed all = 0.042058052 MB/sec (without seq num) 2024/11/20-18:28:14.919843 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56033: 129435 keys, 41694811 bytes, temperature: kWarm 2024/11/20-18:28:14.919912 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098494919880, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56033, "file_size": 41694811, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40095904, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588700, "raw_average_key_size": 20, "raw_value_size": 131629059, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129435, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098493, "oldest_key_time": 0, "file_creation_time": 1732098493, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56033, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:14.958171 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056034.sst) /dev/shm/ToplingTemp/Topling-806427-O8fXEs 2024/11/20-18:28:15.601222 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056034.sst): first pass time = 0.64's, 208.735'MB/sec 2024/11/20-18:28:16.154267 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056034.sst): old prealloc_size = 147639500, real_size = 40349952 2024/11/20-18:28:16.154397 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056034.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129432 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.98 avg-zval = 309.80 2024/11/20-18:28:16.154430 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056034.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:16.322066 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056034.sst): second pass time = 0.55's, 238.301'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 248.542'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2248.972'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.267'MB/sec write SST data time = 0.05's, 897.281'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 50091.953'MB/sec zip my value time = 0.55's, unzip length = 0.132'GB zip my value throughput = 238.301'MB/sec zip pipeline throughput = 238.301'MB/sec entries = 129432 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.98 avg-zval = 309.80 usrkeys = 129432 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.98 avg-zval = 309.80 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0981 value = 3.2827 dict = 1.00 all = 3.2189 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 1.579555 GB avg = 1.017 KB (by entry num) total key len = 0.031063 GB avg = 0.020 KB total ukey len = 0.018638 GB avg = 0.012 KB total ukey num = 0.001553155 Billion total entry num = 0.001553155 Billion write speed all = 0.046236306 MB/sec (with seq num) write speed all = 0.045879612 MB/sec (without seq num) 2024/11/20-18:28:16.377921 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56034: 129432 keys, 41697512 bytes, temperature: kWarm 2024/11/20-18:28:16.378000 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098496377955, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56034, "file_size": 41697512, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40098608, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588640, "raw_average_key_size": 20, "raw_value_size": 131629944, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098494, "oldest_key_time": 0, "file_creation_time": 1732098494, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56034, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:16.418631 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056035.sst) /dev/shm/ToplingTemp/Topling-806427-ruCNxo 2024/11/20-18:28:17.040341 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056035.sst): first pass time = 0.62's, 215.902'MB/sec 2024/11/20-18:28:17.584734 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056035.sst): old prealloc_size = 147639500, real_size = 40353968 2024/11/20-18:28:17.584848 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056035.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.01 avg-zval = 309.85 2024/11/20-18:28:17.584883 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056035.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:17.790161 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056035.sst): second pass time = 0.54's, 242.036'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.21's, 203.034'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2327.533'MB/sec (index lex order gen) rebuild zvType time = 0.15's, 0.219'MB/sec write SST data time = 0.06's, 727.053'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 32540.216'MB/sec zip my value time = 0.54's, unzip length = 0.132'GB zip my value throughput = 242.036'MB/sec zip pipeline throughput = 242.036'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.85 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.85 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2823 dict = 1.00 all = 3.2186 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 1.711184 GB avg = 1.017 KB (by entry num) total key len = 0.033652 GB avg = 0.020 KB total ukey len = 0.020191 GB avg = 0.012 KB total ukey num = 0.001682583 Billion total entry num = 0.001682583 Billion write speed all = 0.050087218 MB/sec (with seq num) write speed all = 0.049700816 MB/sec (without seq num) 2024/11/20-18:28:17.867868 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56035: 129428 keys, 41701262 bytes, temperature: kWarm 2024/11/20-18:28:17.867927 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098497867899, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56035, "file_size": 41701262, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40102624, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629497, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098496, "oldest_key_time": 0, "file_creation_time": 1732098496, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56035, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:17.908081 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056036.sst) /dev/shm/ToplingTemp/Topling-806427-7clPHr 2024/11/20-18:28:18.498367 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056036.sst): first pass time = 0.59's, 227.390'MB/sec 2024/11/20-18:28:19.067380 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056036.sst): old prealloc_size = 147639500, real_size = 40352208 2024/11/20-18:28:19.067492 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056036.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.00 avg-zval = 309.83 2024/11/20-18:28:19.067519 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056036.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:19.273666 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056036.sst): second pass time = 0.56's, 235.904'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.21's, 202.157'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2433.614'MB/sec (index lex order gen) rebuild zvType time = 0.16's, 0.205'MB/sec write SST data time = 0.05's, 861.834'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44037.010'MB/sec zip my value time = 0.56's, unzip length = 0.132'GB zip my value throughput = 235.904'MB/sec zip pipeline throughput = 235.904'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.83 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.83 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2825 dict = 1.00 all = 3.2187 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 1.842814 GB avg = 1.017 KB (by entry num) total key len = 0.036240 GB avg = 0.020 KB total ukey len = 0.021744 GB avg = 0.012 KB total ukey num = 0.001812013 Billion total entry num = 0.001812013 Billion write speed all = 0.053937797 MB/sec (with seq num) write speed all = 0.053521690 MB/sec (without seq num) 2024/11/20-18:28:19.325183 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56036: 129430 keys, 41699396 bytes, temperature: kWarm 2024/11/20-18:28:19.325241 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098499325211, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56036, "file_size": 41699396, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40100864, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131630022, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098497, "oldest_key_time": 0, "file_creation_time": 1732098497, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56036, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:19.366519 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056037.sst) /dev/shm/ToplingTemp/Topling-806427-u8IC5q 2024/11/20-18:28:19.910612 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056037.sst): first pass time = 0.54's, 246.696'MB/sec 2024/11/20-18:28:20.481312 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056037.sst): old prealloc_size = 147639500, real_size = 40351664 2024/11/20-18:28:20.481424 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056037.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.01 avg-zval = 309.83 2024/11/20-18:28:20.481452 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056037.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:20.646167 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056037.sst): second pass time = 0.57's, 230.886'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 252.945'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2427.620'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.267'MB/sec write SST data time = 0.04's, 959.821'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 47536.898'MB/sec zip my value time = 0.57's, unzip length = 0.132'GB zip my value throughput = 230.886'MB/sec zip pipeline throughput = 230.898'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.83 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.83 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2825 dict = 1.00 all = 3.2187 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 1.974444 GB avg = 1.017 KB (by entry num) total key len = 0.038829 GB avg = 0.020 KB total ukey len = 0.023297 GB avg = 0.012 KB total ukey num = 0.001941441 Billion total entry num = 0.001941441 Billion write speed all = 0.057788213 MB/sec (with seq num) write speed all = 0.057342402 MB/sec (without seq num) 2024/11/20-18:28:20.696138 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56037: 129428 keys, 41698867 bytes, temperature: kWarm 2024/11/20-18:28:20.696189 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098500696161, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56037, "file_size": 41698867, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40100320, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629356, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098499, "oldest_key_time": 0, "file_creation_time": 1732098499, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56037, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:20.734949 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056038.sst) /dev/shm/ToplingTemp/Topling-806427-JukZzr 2024/11/20-18:28:21.352700 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056038.sst): first pass time = 0.62's, 217.280'MB/sec 2024/11/20-18:28:21.878776 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056038.sst): old prealloc_size = 147639500, real_size = 40351680 2024/11/20-18:28:21.878898 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056038.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.00 avg-zval = 309.82 2024/11/20-18:28:21.878934 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056038.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:22.039595 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056038.sst): second pass time = 0.53's, 250.544'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 259.317'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2202.343'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.279'MB/sec write SST data time = 0.04's, 929.555'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 25400.015'MB/sec zip my value time = 0.53's, unzip length = 0.132'GB zip my value throughput = 250.544'MB/sec zip pipeline throughput = 250.544'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.82 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.82 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2825 dict = 1.00 all = 3.2187 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 2.106073 GB avg = 1.017 KB (by entry num) total key len = 0.041417 GB avg = 0.020 KB total ukey len = 0.024850 GB avg = 0.012 KB total ukey num = 0.002070870 Billion total entry num = 0.002070870 Billion write speed all = 0.061638287 MB/sec (with seq num) write speed all = 0.061162774 MB/sec (without seq num) 2024/11/20-18:28:22.088996 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56038: 129429 keys, 41698910 bytes, temperature: kWarm 2024/11/20-18:28:22.089080 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098502089024, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56038, "file_size": 41698910, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40100336, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629298, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098500, "oldest_key_time": 0, "file_creation_time": 1732098500, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56038, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:22.127474 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056039.sst) /dev/shm/ToplingTemp/Topling-806427-8BGvxo 2024/11/20-18:28:22.818301 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056039.sst): first pass time = 0.68's, 196.171'MB/sec 2024/11/20-18:28:23.332706 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056039.sst): old prealloc_size = 147639500, real_size = 40349920 2024/11/20-18:28:23.332834 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056039.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129432 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.98 avg-zval = 309.80 2024/11/20-18:28:23.332871 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056039.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:23.497318 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056039.sst): second pass time = 0.51's, 256.281'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 253.341'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2057.179'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.283'MB/sec write SST data time = 0.05's, 832.366'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 25413.623'MB/sec zip my value time = 0.51's, unzip length = 0.132'GB zip my value throughput = 256.281'MB/sec zip pipeline throughput = 256.281'MB/sec entries = 129432 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.98 avg-zval = 309.80 usrkeys = 129432 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.98 avg-zval = 309.80 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0981 value = 3.2826 dict = 1.00 all = 3.2189 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 2.237702 GB avg = 1.017 KB (by entry num) total key len = 0.044006 GB avg = 0.020 KB total ukey len = 0.026404 GB avg = 0.012 KB total ukey num = 0.002200302 Billion total entry num = 0.002200302 Billion write speed all = 0.065487935 MB/sec (with seq num) write speed all = 0.064982723 MB/sec (without seq num) 2024/11/20-18:28:23.547522 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56039: 129432 keys, 41697043 bytes, temperature: kWarm 2024/11/20-18:28:23.547588 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098503547558, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56039, "file_size": 41697043, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40098576, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588640, "raw_average_key_size": 20, "raw_value_size": 131629339, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098502, "oldest_key_time": 0, "file_creation_time": 1732098502, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56039, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:23.584148 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056040.sst) /dev/shm/ToplingTemp/Topling-806427-HPzV4o 2024/11/20-18:28:24.262252 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056040.sst): first pass time = 0.68's, 197.941'MB/sec 2024/11/20-18:28:24.759953 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056040.sst): old prealloc_size = 147639500, real_size = 40348768 2024/11/20-18:28:24.760115 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056040.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.99 avg-zval = 309.80 2024/11/20-18:28:24.760162 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056040.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:24.925882 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056040.sst): second pass time = 0.50's, 264.877'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 251.319'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1493.054'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.269'MB/sec write SST data time = 0.05's, 916.848'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 25062.220'MB/sec zip my value time = 0.50's, unzip length = 0.132'GB zip my value throughput = 264.877'MB/sec zip pipeline throughput = 264.877'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.80 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.80 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2827 dict = 1.00 all = 3.2189 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 2.369332 GB avg = 1.017 KB (by entry num) total key len = 0.046595 GB avg = 0.020 KB total ukey len = 0.027957 GB avg = 0.012 KB total ukey num = 0.002329732 Billion total entry num = 0.002329732 Billion write speed all = 0.069337321 MB/sec (with seq num) write speed all = 0.068802413 MB/sec (without seq num) 2024/11/20-18:28:24.977379 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56040: 129430 keys, 41696438 bytes, temperature: kWarm 2024/11/20-18:28:24.977444 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098504977414, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56040, "file_size": 41696438, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40097424, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629449, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098503, "oldest_key_time": 0, "file_creation_time": 1732098503, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56040, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:25.016117 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056041.sst) /dev/shm/ToplingTemp/Topling-806427-y45dtq 2024/11/20-18:28:25.713000 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056041.sst): first pass time = 0.70's, 192.608'MB/sec 2024/11/20-18:28:26.268889 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056041.sst): old prealloc_size = 147639500, real_size = 40353216 2024/11/20-18:28:26.269027 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056041.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.02 avg-zval = 309.84 2024/11/20-18:28:26.269100 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056041.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:26.436499 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056041.sst): second pass time = 0.56's, 237.167'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 248.810'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2093.132'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.269'MB/sec write SST data time = 0.05's, 886.234'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45193.999'MB/sec zip my value time = 0.56's, unzip length = 0.132'GB zip my value throughput = 237.167'MB/sec zip pipeline throughput = 237.167'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.84 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.84 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2824 dict = 1.00 all = 3.2186 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 2.500962 GB avg = 1.017 KB (by entry num) total key len = 0.049183 GB avg = 0.020 KB total ukey len = 0.029510 GB avg = 0.012 KB total ukey num = 0.002459159 Billion total entry num = 0.002459159 Billion write speed all = 0.073186233 MB/sec (with seq num) write speed all = 0.072621633 MB/sec (without seq num) 2024/11/20-18:28:26.488846 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56041: 129427 keys, 41700737 bytes, temperature: kWarm 2024/11/20-18:28:26.488907 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098506488876, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56041, "file_size": 41700737, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40101872, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629998, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098505, "oldest_key_time": 0, "file_creation_time": 1732098505, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56041, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:26.530657 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056042.sst) /dev/shm/ToplingTemp/Topling-806427-Jus2Qo 2024/11/20-18:28:27.227386 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056042.sst): first pass time = 0.70's, 192.652'MB/sec 2024/11/20-18:28:27.765642 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056042.sst): old prealloc_size = 147639500, real_size = 40351952 2024/11/20-18:28:27.765771 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056042.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.99 avg-zval = 309.82 2024/11/20-18:28:27.765806 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056042.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:27.944587 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056042.sst): second pass time = 0.54's, 244.853'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 233.049'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2109.294'MB/sec (index lex order gen) rebuild zvType time = 0.13's, 0.250'MB/sec write SST data time = 0.05's, 844.094'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 47342.551'MB/sec zip my value time = 0.54's, unzip length = 0.132'GB zip my value throughput = 244.853'MB/sec zip pipeline throughput = 244.853'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.82 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.82 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2825 dict = 1.00 all = 3.2187 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 2.632591 GB avg = 1.017 KB (by entry num) total key len = 0.051772 GB avg = 0.020 KB total ukey len = 0.031063 GB avg = 0.012 KB total ukey num = 0.002588589 Billion total entry num = 0.002588589 Billion write speed all = 0.077034805 MB/sec (with seq num) write speed all = 0.076440514 MB/sec (without seq num) 2024/11/20-18:28:27.997095 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56042: 129430 keys, 41699455 bytes, temperature: kWarm 2024/11/20-18:28:27.997154 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098507997124, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56042, "file_size": 41699455, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40100608, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629505, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098506, "oldest_key_time": 0, "file_creation_time": 1732098506, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56042, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:28.044982 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056043.sst) /dev/shm/ToplingTemp/Topling-806427-vh6C4r 2024/11/20-18:28:28.661095 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056043.sst): first pass time = 0.62's, 217.863'MB/sec 2024/11/20-18:28:29.225133 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056043.sst): old prealloc_size = 147639500, real_size = 40354032 2024/11/20-18:28:29.225264 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056043.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129424 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.05 avg-zval = 309.86 2024/11/20-18:28:29.225312 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056043.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:29.414000 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056043.sst): second pass time = 0.56's, 233.636'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.19's, 220.806'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2042.204'MB/sec (index lex order gen) rebuild zvType time = 0.14's, 0.237'MB/sec write SST data time = 0.05's, 799.882'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 35944.075'MB/sec zip my value time = 0.56's, unzip length = 0.132'GB zip my value throughput = 233.636'MB/sec zip pipeline throughput = 233.636'MB/sec entries = 129424 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.05 avg-zval = 309.86 usrkeys = 129424 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.05 avg-zval = 309.86 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0977 value = 3.2823 dict = 1.00 all = 3.2186 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 2.764221 GB avg = 1.017 KB (by entry num) total key len = 0.054360 GB avg = 0.020 KB total ukey len = 0.032616 GB avg = 0.012 KB total ukey num = 0.002718013 Billion total entry num = 0.002718013 Billion write speed all = 0.080883152 MB/sec (with seq num) write speed all = 0.080259174 MB/sec (without seq num) 2024/11/20-18:28:29.468208 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56043: 129424 keys, 41701366 bytes, temperature: kWarm 2024/11/20-18:28:29.468288 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098509468244, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56043, "file_size": 41701366, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40102688, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588480, "raw_average_key_size": 20, "raw_value_size": 131630269, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129424, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098508, "oldest_key_time": 0, "file_creation_time": 1732098508, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56043, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:29.521315 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056044.sst) /dev/shm/ToplingTemp/Topling-806427-JdyS1q 2024/11/20-18:28:30.070033 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056044.sst): first pass time = 0.55's, 244.619'MB/sec 2024/11/20-18:28:30.630229 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056044.sst): old prealloc_size = 147639500, real_size = 40344272 2024/11/20-18:28:30.630358 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056044.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129438 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.93 avg-zval = 309.75 2024/11/20-18:28:30.630394 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056044.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:30.813899 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056044.sst): second pass time = 0.56's, 235.233'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 227.002'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2056.889'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.260'MB/sec write SST data time = 0.06's, 703.394'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 22595.612'MB/sec zip my value time = 0.56's, unzip length = 0.132'GB zip my value throughput = 235.233'MB/sec zip pipeline throughput = 235.233'MB/sec entries = 129438 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.93 avg-zval = 309.75 usrkeys = 129438 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.93 avg-zval = 309.75 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0985 value = 3.2831 dict = 1.00 all = 3.2193 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3106 } ---------------------------- total value len = 2.895851 GB avg = 1.017 KB (by entry num) total key len = 0.056949 GB avg = 0.020 KB total ukey len = 0.034169 GB avg = 0.012 KB total ukey num = 0.002847451 Billion total entry num = 0.002847451 Billion write speed all = 0.084731321 MB/sec (with seq num) write speed all = 0.084077654 MB/sec (without seq num) 2024/11/20-18:28:30.865485 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56044: 129438 keys, 41691715 bytes, temperature: kWarm 2024/11/20-18:28:30.865544 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098510865512, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56044, "file_size": 41691715, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40092928, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588760, "raw_average_key_size": 20, "raw_value_size": 131629212, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129438, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098509, "oldest_key_time": 0, "file_creation_time": 1732098509, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56044, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:30.907821 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056045.sst) /dev/shm/ToplingTemp/Topling-806427-SZp3mo 2024/11/20-18:28:31.440386 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056045.sst): first pass time = 0.53's, 252.040'MB/sec 2024/11/20-18:28:32.022642 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056045.sst): old prealloc_size = 147639500, real_size = 40353696 2024/11/20-18:28:32.022772 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056045.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.01 avg-zval = 309.84 2024/11/20-18:28:32.022816 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056045.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:32.191340 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056045.sst): second pass time = 0.58's, 226.362'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 247.204'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2167.203'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.268'MB/sec write SST data time = 0.05's, 873.805'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 49068.150'MB/sec zip my value time = 0.58's, unzip length = 0.132'GB zip my value throughput = 226.362'MB/sec zip pipeline throughput = 226.362'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.84 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.84 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2823 dict = 1.00 all = 3.2186 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 3.027481 GB avg = 1.017 KB (by entry num) total key len = 0.059538 GB avg = 0.020 KB total ukey len = 0.035723 GB avg = 0.012 KB total ukey num = 0.002976879 Billion total entry num = 0.002976879 Billion write speed all = 0.088579250 MB/sec (with seq num) write speed all = 0.087895899 MB/sec (without seq num) 2024/11/20-18:28:32.248872 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56045: 129428 keys, 41701073 bytes, temperature: kWarm 2024/11/20-18:28:32.248974 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098512248922, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56045, "file_size": 41701073, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40102352, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629867, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098510, "oldest_key_time": 0, "file_creation_time": 1732098510, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56045, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:32.306709 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056046.sst) /dev/shm/ToplingTemp/Topling-806427-fKnTGs 2024/11/20-18:28:32.985997 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056046.sst): first pass time = 0.68's, 197.600'MB/sec 2024/11/20-18:28:33.529107 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056046.sst): old prealloc_size = 147639500, real_size = 40354048 2024/11/20-18:28:33.529227 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056046.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.01 avg-zval = 309.85 2024/11/20-18:28:33.529256 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056046.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:33.698488 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056046.sst): second pass time = 0.54's, 242.675'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 246.227'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2230.859'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.269'MB/sec write SST data time = 0.05's, 848.883'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 35897.080'MB/sec zip my value time = 0.54's, unzip length = 0.132'GB zip my value throughput = 242.675'MB/sec zip pipeline throughput = 248.240'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.85 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.85 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2823 dict = 1.00 all = 3.2186 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 3.159110 GB avg = 1.017 KB (by entry num) total key len = 0.062126 GB avg = 0.020 KB total ukey len = 0.037276 GB avg = 0.012 KB total ukey num = 0.003106307 Billion total entry num = 0.003106307 Billion write speed all = 0.092426533 MB/sec (with seq num) write speed all = 0.091713502 MB/sec (without seq num) 2024/11/20-18:28:33.750555 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56046: 129428 keys, 41701431 bytes, temperature: kWarm 2024/11/20-18:28:33.750626 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098513750594, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56046, "file_size": 41701431, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40102704, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629904, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098512, "oldest_key_time": 0, "file_creation_time": 1732098512, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56046, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:33.790015 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056047.sst) /dev/shm/ToplingTemp/Topling-806427-mpNw4p 2024/11/20-18:28:34.332640 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056047.sst): first pass time = 0.54's, 247.368'MB/sec 2024/11/20-18:28:35.638504 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056047.sst): old prealloc_size = 147639500, real_size = 40349488 2024/11/20-18:28:35.638620 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056047.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129432 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.98 avg-zval = 309.80 2024/11/20-18:28:35.638648 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056047.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:35.805445 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056047.sst): second pass time = 1.31's, 100.837'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 249.795'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2283.877'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.270'MB/sec write SST data time = 0.05's, 890.019'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 31261.306'MB/sec zip my value time = 1.31's, unzip length = 0.132'GB zip my value throughput = 100.837'MB/sec zip pipeline throughput = 200.643'MB/sec entries = 129432 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.98 avg-zval = 309.80 usrkeys = 129432 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.98 avg-zval = 309.80 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0981 value = 3.2827 dict = 1.00 all = 3.2189 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 3.290740 GB avg = 1.017 KB (by entry num) total key len = 0.064715 GB avg = 0.020 KB total ukey len = 0.038829 GB avg = 0.012 KB total ukey num = 0.003235739 Billion total entry num = 0.003235739 Billion write speed all = 0.096271813 MB/sec (with seq num) write speed all = 0.095529117 MB/sec (without seq num) 2024/11/20-18:28:35.859911 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56047: 129432 keys, 41696551 bytes, temperature: kWarm 2024/11/20-18:28:35.859968 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098515859938, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56047, "file_size": 41696551, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40098144, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588640, "raw_average_key_size": 20, "raw_value_size": 131629348, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098513, "oldest_key_time": 0, "file_creation_time": 1732098513, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56047, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:35.905978 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056048.sst) /dev/shm/ToplingTemp/Topling-806427-ClJfYq 2024/11/20-18:28:36.528101 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056048.sst): first pass time = 0.62's, 215.756'MB/sec 2024/11/20-18:28:37.126487 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056048.sst): old prealloc_size = 147639500, real_size = 40347648 2024/11/20-18:28:37.126612 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056048.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129434 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.96 avg-zval = 309.78 2024/11/20-18:28:37.126648 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056048.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:37.300139 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056048.sst): second pass time = 0.60's, 220.235'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 240.123'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2026.599'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.262'MB/sec write SST data time = 0.05's, 832.616'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 47529.939'MB/sec zip my value time = 0.60's, unzip length = 0.132'GB zip my value throughput = 220.235'MB/sec zip pipeline throughput = 220.235'MB/sec entries = 129434 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.96 avg-zval = 309.78 usrkeys = 129434 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.96 avg-zval = 309.78 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0982 value = 3.2828 dict = 1.00 all = 3.2191 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 3.422369 GB avg = 1.017 KB (by entry num) total key len = 0.067303 GB avg = 0.020 KB total ukey len = 0.040382 GB avg = 0.012 KB total ukey num = 0.003365173 Billion total entry num = 0.003365173 Billion write speed all = 0.100118395 MB/sec (with seq num) write speed all = 0.099346023 MB/sec (without seq num) 2024/11/20-18:28:37.353992 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56048: 129434 keys, 41694966 bytes, temperature: kWarm 2024/11/20-18:28:37.354095 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098517354039, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56048, "file_size": 41694966, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40096304, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588680, "raw_average_key_size": 20, "raw_value_size": 131629624, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129434, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098515, "oldest_key_time": 0, "file_creation_time": 1732098515, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56048, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:37.414072 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056049.sst) /dev/shm/ToplingTemp/Topling-806427-fEBg7r 2024/11/20-18:28:38.112693 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056049.sst): first pass time = 0.70's, 192.129'MB/sec 2024/11/20-18:28:38.916902 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056049.sst): old prealloc_size = 147639500, real_size = 40348304 2024/11/20-18:28:38.917030 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056049.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129434 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.96 avg-zval = 309.79 2024/11/20-18:28:38.917098 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056049.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:39.089846 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056049.sst): second pass time = 0.80's, 163.835'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 241.238'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1837.215'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.267'MB/sec write SST data time = 0.05's, 808.385'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 16279.445'MB/sec zip my value time = 0.80's, unzip length = 0.132'GB zip my value throughput = 163.835'MB/sec zip pipeline throughput = 200.840'MB/sec entries = 129434 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.96 avg-zval = 309.79 usrkeys = 129434 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.96 avg-zval = 309.79 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 9 bytes = 16 } seq expand size = 63 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0982 value = 3.2828 dict = 1.00 all = 3.2174 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 3.553999 GB avg = 1.017 KB (by entry num) total key len = 0.069892 GB avg = 0.020 KB total ukey len = 0.041935 GB avg = 0.012 KB total ukey num = 0.003494607 Billion total entry num = 0.003494607 Billion write speed all = 0.103963755 MB/sec (with seq num) write speed all = 0.103161716 MB/sec (without seq num) 2024/11/20-18:28:39.191283 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56049: 129434 keys, 41716152 bytes, temperature: kWarm 2024/11/20-18:28:39.191362 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098519191329, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56049, "file_size": 41716152, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 939480227, "table_properties": {"data_size": 40096960, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588680, "raw_average_key_size": 20, "raw_value_size": 131629102, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129434, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098517, "oldest_key_time": 0, "file_creation_time": 1732098517, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56049, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:39.230592 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056050.sst) /dev/shm/ToplingTemp/Topling-806427-6qYlMq 2024/11/20-18:28:39.776765 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056050.sst): first pass time = 0.55's, 245.762'MB/sec 2024/11/20-18:28:40.348966 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056050.sst): old prealloc_size = 147639500, real_size = 40352000 2024/11/20-18:28:40.349129 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056050.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.01 avg-zval = 309.83 2024/11/20-18:28:40.349188 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056050.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:40.559237 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056050.sst): second pass time = 0.57's, 230.357'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.21's, 198.448'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1585.233'MB/sec (index lex order gen) rebuild zvType time = 0.14's, 0.226'MB/sec write SST data time = 0.07's, 621.777'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 22711.853'MB/sec zip my value time = 0.57's, unzip length = 0.132'GB zip my value throughput = 230.357'MB/sec zip pipeline throughput = 230.357'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.83 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.83 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 1 bytes = 16 } seq expand size = 7 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2825 dict = 1.00 all = 3.2172 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 3.685629 GB avg = 1.017 KB (by entry num) total key len = 0.072481 GB avg = 0.020 KB total ukey len = 0.043488 GB avg = 0.012 KB total ukey num = 0.003624036 Billion total entry num = 0.003624036 Billion write speed all = 0.107809735 MB/sec (with seq num) write speed all = 0.106978027 MB/sec (without seq num) 2024/11/20-18:28:40.611765 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56050: 129429 keys, 41719396 bytes, temperature: kWarm 2024/11/20-18:28:40.611832 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098520611802, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56050, "file_size": 41719396, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 939480228, "table_properties": {"data_size": 40100656, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131630117, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098519, "oldest_key_time": 0, "file_creation_time": 1732098519, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56050, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:40.656580 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056051.sst) /dev/shm/ToplingTemp/Topling-806427-6KWfVp 2024/11/20-18:28:41.663811 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056051.sst): first pass time = 1.01's, 133.260'MB/sec 2024/11/20-18:28:42.264422 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056051.sst): old prealloc_size = 147639500, real_size = 40353456 2024/11/20-18:28:42.264537 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056051.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.00 avg-zval = 309.84 2024/11/20-18:28:42.264574 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056051.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:42.441948 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056051.sst): second pass time = 0.60's, 219.412'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 234.914'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2316.935'MB/sec (index lex order gen) rebuild zvType time = 0.13's, 0.247'MB/sec write SST data time = 0.05's, 897.266'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 49093.426'MB/sec zip my value time = 0.60's, unzip length = 0.132'GB zip my value throughput = 219.412'MB/sec zip pipeline throughput = 219.412'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.84 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.84 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2824 dict = 1.00 all = 3.2186 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 3.817258 GB avg = 1.017 KB (by entry num) total key len = 0.075069 GB avg = 0.020 KB total ukey len = 0.045042 GB avg = 0.012 KB total ukey num = 0.003753466 Billion total entry num = 0.003753466 Billion write speed all = 0.111654055 MB/sec (with seq num) write speed all = 0.110792689 MB/sec (without seq num) 2024/11/20-18:28:42.492721 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56051: 129430 keys, 41700609 bytes, temperature: kWarm 2024/11/20-18:28:42.492791 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098522492757, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56051, "file_size": 41700609, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40102112, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629698, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098520, "oldest_key_time": 0, "file_creation_time": 1732098520, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56051, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:42.526769 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056052.sst) /dev/shm/ToplingTemp/Topling-806427-pY19Lq 2024/11/20-18:28:43.044836 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056052.sst): first pass time = 0.52's, 259.101'MB/sec 2024/11/20-18:28:43.544350 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056052.sst): old prealloc_size = 147639500, real_size = 40353072 2024/11/20-18:28:43.544479 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056052.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.00 avg-zval = 309.84 2024/11/20-18:28:43.544521 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056052.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:43.713117 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056052.sst): second pass time = 0.50's, 263.997'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 247.070'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2266.584'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.268'MB/sec write SST data time = 0.05's, 872.006'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 26549.802'MB/sec zip my value time = 0.50's, unzip length = 0.132'GB zip my value throughput = 263.997'MB/sec zip pipeline throughput = 263.997'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.84 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.84 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2824 dict = 1.00 all = 3.2187 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 3.948888 GB avg = 1.017 KB (by entry num) total key len = 0.077658 GB avg = 0.020 KB total ukey len = 0.046595 GB avg = 0.012 KB total ukey num = 0.003882895 Billion total entry num = 0.003882895 Billion write speed all = 0.115499992 MB/sec (with seq num) write speed all = 0.114608956 MB/sec (without seq num) 2024/11/20-18:28:43.763572 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56052: 129429 keys, 41700220 bytes, temperature: kWarm 2024/11/20-18:28:43.763637 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098523763607, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56052, "file_size": 41700220, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40101728, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629911, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098522, "oldest_key_time": 0, "file_creation_time": 1732098522, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56052, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:43.799900 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056053.sst) /dev/shm/ToplingTemp/Topling-806427-EFwyYp 2024/11/20-18:28:44.762592 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056053.sst): first pass time = 0.96's, 139.425'MB/sec 2024/11/20-18:28:45.401903 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056053.sst): old prealloc_size = 147639500, real_size = 40352192 2024/11/20-18:28:45.402024 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056053.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129426 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.02 avg-zval = 309.84 2024/11/20-18:28:45.402083 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056053.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:45.572432 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056053.sst): second pass time = 0.64's, 206.114'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 244.559'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2110.494'MB/sec (index lex order gen) rebuild zvType time = 0.13's, 0.258'MB/sec write SST data time = 0.05's, 925.245'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 46658.790'MB/sec zip my value time = 0.64's, unzip length = 0.132'GB zip my value throughput = 206.114'MB/sec zip pipeline throughput = 206.114'MB/sec entries = 129426 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.84 usrkeys = 129426 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.84 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0978 value = 3.2825 dict = 1.00 all = 3.2187 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 4.080518 GB avg = 1.017 KB (by entry num) total key len = 0.080246 GB avg = 0.020 KB total ukey len = 0.048148 GB avg = 0.012 KB total ukey num = 0.004012321 Billion total entry num = 0.004012321 Billion write speed all = 0.119343618 MB/sec (with seq num) write speed all = 0.118422931 MB/sec (without seq num) 2024/11/20-18:28:45.625911 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56053: 129426 keys, 41699776 bytes, temperature: kWarm 2024/11/20-18:28:45.625963 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098525625937, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56053, "file_size": 41699776, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40100848, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588520, "raw_average_key_size": 20, "raw_value_size": 131629393, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129426, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098523, "oldest_key_time": 0, "file_creation_time": 1732098523, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56053, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:45.666693 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056054.sst) /dev/shm/ToplingTemp/Topling-806427-lUVGep 2024/11/20-18:28:46.209598 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056054.sst): first pass time = 0.54's, 247.236'MB/sec 2024/11/20-18:28:46.797744 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056054.sst): old prealloc_size = 147639500, real_size = 40350064 2024/11/20-18:28:46.797865 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056054.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129433 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.97 avg-zval = 309.80 2024/11/20-18:28:46.797901 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056054.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:46.967419 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056054.sst): second pass time = 0.59's, 224.048'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 245.765'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2218.037'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.260'MB/sec write SST data time = 0.04's, 928.157'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 32888.820'MB/sec zip my value time = 0.59's, unzip length = 0.132'GB zip my value throughput = 224.048'MB/sec zip pipeline throughput = 224.048'MB/sec entries = 129433 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.97 avg-zval = 309.80 usrkeys = 129433 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.97 avg-zval = 309.80 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0982 value = 3.2826 dict = 1.00 all = 3.2189 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 4.212147 GB avg = 1.017 KB (by entry num) total key len = 0.082835 GB avg = 0.020 KB total ukey len = 0.049701 GB avg = 0.012 KB total ukey num = 0.004141754 Billion total entry num = 0.004141754 Billion write speed all = 0.123188486 MB/sec (with seq num) write speed all = 0.122238137 MB/sec (without seq num) 2024/11/20-18:28:47.019930 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56054: 129433 keys, 41697320 bytes, temperature: kWarm 2024/11/20-18:28:47.019997 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098527019967, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56054, "file_size": 41697320, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40098720, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588660, "raw_average_key_size": 20, "raw_value_size": 131629651, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129433, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098525, "oldest_key_time": 0, "file_creation_time": 1732098525, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56054, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:47.058598 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056055.sst) /dev/shm/ToplingTemp/Topling-806427-JMlnoo 2024/11/20-18:28:47.899880 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056055.sst): first pass time = 0.84's, 159.547'MB/sec 2024/11/20-18:28:48.529264 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056055.sst): old prealloc_size = 147639500, real_size = 40355264 2024/11/20-18:28:48.529445 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056055.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.01 avg-zval = 309.85 2024/11/20-18:28:48.529543 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056055.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:48.734907 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056055.sst): second pass time = 0.61's, 216.158'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.21's, 202.897'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1585.910'MB/sec (index lex order gen) rebuild zvType time = 0.15's, 0.209'MB/sec write SST data time = 0.05's, 823.663'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.379'MB, throughput = 43494.258'MB/sec zip my value time = 0.61's, unzip length = 0.132'GB zip my value throughput = 216.158'MB/sec zip pipeline throughput = 249.940'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.85 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.85 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 1 bytes = 16 } seq expand size = 7 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2822 dict = 1.00 all = 3.2169 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 4.343777 GB avg = 1.017 KB (by entry num) total key len = 0.085424 GB avg = 0.020 KB total ukey len = 0.051254 GB avg = 0.012 KB total ukey num = 0.004271183 Billion total entry num = 0.004271183 Billion write speed all = 0.127031701 MB/sec (with seq num) write speed all = 0.126051703 MB/sec (without seq num) 2024/11/20-18:28:48.786566 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56055: 129429 keys, 41723466 bytes, temperature: kWarm 2024/11/20-18:28:48.786626 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098528786595, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56055, "file_size": 41723466, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 939521583, "table_properties": {"data_size": 40103920, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131630109, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098527, "oldest_key_time": 0, "file_creation_time": 1732098527, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56055, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:48.837795 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056056.sst) /dev/shm/ToplingTemp/Topling-806427-l90fCp 2024/11/20-18:28:49.444598 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056056.sst): first pass time = 0.61's, 221.203'MB/sec 2024/11/20-18:28:50.004193 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056056.sst): old prealloc_size = 147639500, real_size = 40350624 2024/11/20-18:28:50.004323 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056056.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129432 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.98 avg-zval = 309.81 2024/11/20-18:28:50.004379 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056056.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:50.171607 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056056.sst): second pass time = 0.56's, 235.516'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 249.206'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2063.173'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.268'MB/sec write SST data time = 0.05's, 896.993'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 34821.559'MB/sec zip my value time = 0.56's, unzip length = 0.132'GB zip my value throughput = 235.516'MB/sec zip pipeline throughput = 235.516'MB/sec entries = 129432 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.98 avg-zval = 309.81 usrkeys = 129432 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.98 avg-zval = 309.81 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 76 bytes = 160 } seq expand size = 532 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0981 value = 3.2826 dict = 1.00 all = 3.2172 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 4.475408 GB avg = 1.017 KB (by entry num) total key len = 0.088012 GB avg = 0.020 KB total ukey len = 0.052807 GB avg = 0.012 KB total ukey num = 0.004400615 Billion total entry num = 0.004400615 Billion write speed all = 0.130875766 MB/sec (with seq num) write speed all = 0.129866113 MB/sec (without seq num) 2024/11/20-18:28:50.223698 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56056: 129432 keys, 41718577 bytes, temperature: kWarm 2024/11/20-18:28:50.223753 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098530223724, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56056, "file_size": 41718577, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 939521612, "table_properties": {"data_size": 40099280, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588640, "raw_average_key_size": 20, "raw_value_size": 131629497, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098528, "oldest_key_time": 0, "file_creation_time": 1732098528, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56056, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:50.261767 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056057.sst) /dev/shm/ToplingTemp/Topling-806427-pyne9r 2024/11/20-18:28:51.000890 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056057.sst): first pass time = 0.74's, 181.598'MB/sec 2024/11/20-18:28:51.599193 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056057.sst): old prealloc_size = 147639500, real_size = 40354112 2024/11/20-18:28:51.599473 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056057.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129426 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.02 avg-zval = 309.85 2024/11/20-18:28:51.599593 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056057.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:51.802719 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056057.sst): second pass time = 0.60's, 220.319'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.20's, 204.965'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 922.581'MB/sec (index lex order gen) rebuild zvType time = 0.16's, 0.206'MB/sec write SST data time = 0.05's, 901.263'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 49507.204'MB/sec zip my value time = 0.60's, unzip length = 0.132'GB zip my value throughput = 220.319'MB/sec zip pipeline throughput = 220.319'MB/sec entries = 129426 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.85 usrkeys = 129426 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.85 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 10 bytes = 32 } seq expand size = 70 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0978 value = 3.2823 dict = 1.00 all = 3.2170 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 4.607037 GB avg = 1.017 KB (by entry num) total key len = 0.090601 GB avg = 0.020 KB total ukey len = 0.054360 GB avg = 0.012 KB total ukey num = 0.004530041 Billion total entry num = 0.004530041 Billion write speed all = 0.134718743 MB/sec (with seq num) write speed all = 0.133679444 MB/sec (without seq num) 2024/11/20-18:28:51.854084 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56057: 129426 keys, 41721706 bytes, temperature: kWarm 2024/11/20-18:28:51.854144 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098531854113, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56057, "file_size": 41721706, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 939521613, "table_properties": {"data_size": 40102768, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588520, "raw_average_key_size": 20, "raw_value_size": 131629341, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129426, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098530, "oldest_key_time": 0, "file_creation_time": 1732098530, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56057, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:51.891087 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056058.sst) /dev/shm/ToplingTemp/Topling-806427-Yh531o 2024/11/20-18:28:52.573060 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056058.sst): first pass time = 0.68's, 196.819'MB/sec 2024/11/20-18:28:53.074794 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056058.sst): old prealloc_size = 147639500, real_size = 40351408 2024/11/20-18:28:53.074908 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056058.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.01 avg-zval = 309.83 2024/11/20-18:28:53.074936 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056058.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:53.241467 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056058.sst): second pass time = 0.50's, 262.677'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 250.214'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2358.134'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.267'MB/sec write SST data time = 0.05's, 921.936'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 25585.391'MB/sec zip my value time = 0.50's, unzip length = 0.132'GB zip my value throughput = 262.677'MB/sec zip pipeline throughput = 277.663'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.83 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.83 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2825 dict = 1.00 all = 3.2187 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 4.738667 GB avg = 1.017 KB (by entry num) total key len = 0.093189 GB avg = 0.020 KB total ukey len = 0.055914 GB avg = 0.012 KB total ukey num = 0.004659469 Billion total entry num = 0.004659469 Billion write speed all = 0.138562148 MB/sec (with seq num) write speed all = 0.137493198 MB/sec (without seq num) 2024/11/20-18:28:53.292602 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56058: 129428 keys, 41699039 bytes, temperature: kWarm 2024/11/20-18:28:53.292658 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098533292627, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56058, "file_size": 41699039, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40100064, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131630173, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098531, "oldest_key_time": 0, "file_creation_time": 1732098531, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56058, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:53.335561 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056059.sst) /dev/shm/ToplingTemp/Topling-806427-wkzubp 2024/11/20-18:28:53.940571 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056059.sst): first pass time = 0.60's, 221.861'MB/sec 2024/11/20-18:28:54.536530 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056059.sst): old prealloc_size = 147639500, real_size = 40350736 2024/11/20-18:28:54.536669 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056059.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129432 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.98 avg-zval = 309.81 2024/11/20-18:28:54.536714 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056059.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:54.721459 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056059.sst): second pass time = 0.60's, 221.150'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 225.514'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1921.292'MB/sec (index lex order gen) rebuild zvType time = 0.14's, 0.239'MB/sec write SST data time = 0.05's, 841.072'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 25615.456'MB/sec zip my value time = 0.60's, unzip length = 0.132'GB zip my value throughput = 221.150'MB/sec zip pipeline throughput = 221.150'MB/sec entries = 129432 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.98 avg-zval = 309.81 usrkeys = 129432 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.98 avg-zval = 309.81 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0981 value = 3.2826 dict = 1.00 all = 3.2188 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 4.870297 GB avg = 1.017 KB (by entry num) total key len = 0.095778 GB avg = 0.020 KB total ukey len = 0.057467 GB avg = 0.012 KB total ukey num = 0.004788901 Billion total entry num = 0.004788901 Billion write speed all = 0.142405064 MB/sec (with seq num) write speed all = 0.141306468 MB/sec (without seq num) 2024/11/20-18:28:54.772984 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56059: 129432 keys, 41698264 bytes, temperature: kWarm 2024/11/20-18:28:54.773058 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098534773016, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56059, "file_size": 41698264, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40099392, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588640, "raw_average_key_size": 20, "raw_value_size": 131629994, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098533, "oldest_key_time": 0, "file_creation_time": 1732098533, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56059, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:54.811485 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056060.sst) /dev/shm/ToplingTemp/Topling-806427-qwZLlq 2024/11/20-18:28:55.409489 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056060.sst): first pass time = 0.60's, 224.455'MB/sec 2024/11/20-18:28:56.031682 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056060.sst): old prealloc_size = 147639500, real_size = 40351232 2024/11/20-18:28:56.031792 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056060.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129432 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.98 avg-zval = 309.81 2024/11/20-18:28:56.031823 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056060.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:56.275867 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056060.sst): second pass time = 0.62's, 211.782'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.24's, 170.793'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2529.402'MB/sec (index lex order gen) rebuild zvType time = 0.18's, 0.184'MB/sec write SST data time = 0.07's, 613.496'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 17281.948'MB/sec zip my value time = 0.62's, unzip length = 0.132'GB zip my value throughput = 211.782'MB/sec zip pipeline throughput = 336.137'MB/sec entries = 129432 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.98 avg-zval = 309.81 usrkeys = 129432 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.98 avg-zval = 309.81 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0981 value = 3.2825 dict = 1.00 all = 3.2188 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 5.001926 GB avg = 1.017 KB (by entry num) total key len = 0.098367 GB avg = 0.020 KB total ukey len = 0.059020 GB avg = 0.012 KB total ukey num = 0.004918333 Billion total entry num = 0.004918333 Billion write speed all = 0.146247324 MB/sec (with seq num) write speed all = 0.145119086 MB/sec (without seq num) 2024/11/20-18:28:56.327690 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56060: 129432 keys, 41698469 bytes, temperature: kWarm 2024/11/20-18:28:56.327751 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098536327720, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56060, "file_size": 41698469, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40099888, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588640, "raw_average_key_size": 20, "raw_value_size": 131629386, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098534, "oldest_key_time": 0, "file_creation_time": 1732098534, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56060, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:56.369181 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056061.sst) /dev/shm/ToplingTemp/Topling-806427-ZJsCSr 2024/11/20-18:28:57.014385 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056061.sst): first pass time = 0.65's, 208.042'MB/sec 2024/11/20-18:28:57.588115 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056061.sst): old prealloc_size = 147639500, real_size = 40349648 2024/11/20-18:28:57.588237 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056061.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.98 avg-zval = 309.80 2024/11/20-18:28:57.588272 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056061.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:57.771202 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056061.sst): second pass time = 0.57's, 229.701'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 227.759'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2225.481'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.266'MB/sec write SST data time = 0.06's, 682.230'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 47697.965'MB/sec zip my value time = 0.57's, unzip length = 0.132'GB zip my value throughput = 229.701'MB/sec zip pipeline throughput = 229.701'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.98 avg-zval = 309.80 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.98 avg-zval = 309.80 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0981 value = 3.2827 dict = 1.00 all = 3.2189 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 5.133556 GB avg = 1.017 KB (by entry num) total key len = 0.100955 GB avg = 0.020 KB total ukey len = 0.060573 GB avg = 0.012 KB total ukey num = 0.005047764 Billion total entry num = 0.005047764 Billion write speed all = 0.150089491 MB/sec (with seq num) write speed all = 0.148931612 MB/sec (without seq num) 2024/11/20-18:28:57.828092 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56061: 129431 keys, 41697147 bytes, temperature: kWarm 2024/11/20-18:28:57.828150 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098537828120, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56061, "file_size": 41697147, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40098304, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629211, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098536, "oldest_key_time": 0, "file_creation_time": 1732098536, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56061, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:57.868452 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056062.sst) /dev/shm/ToplingTemp/Topling-806427-HWbSeo 2024/11/20-18:28:58.502384 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056062.sst): first pass time = 0.63's, 211.737'MB/sec 2024/11/20-18:28:59.024139 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056062.sst): old prealloc_size = 147639500, real_size = 40352912 2024/11/20-18:28:59.024267 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056062.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.01 avg-zval = 309.83 2024/11/20-18:28:59.024304 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056062.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:28:59.271763 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056062.sst): second pass time = 0.52's, 252.605'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.25's, 168.411'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2127.898'MB/sec (index lex order gen) rebuild zvType time = 0.18's, 0.175'MB/sec write SST data time = 0.06's, 666.970'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 49258.885'MB/sec zip my value time = 0.52's, unzip length = 0.132'GB zip my value throughput = 252.605'MB/sec zip pipeline throughput = 252.605'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.83 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.83 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2824 dict = 1.00 all = 3.2187 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 5.265186 GB avg = 1.017 KB (by entry num) total key len = 0.103544 GB avg = 0.020 KB total ukey len = 0.062126 GB avg = 0.012 KB total ukey num = 0.005177193 Billion total entry num = 0.005177193 Billion write speed all = 0.153931332 MB/sec (with seq num) write speed all = 0.152743815 MB/sec (without seq num) 2024/11/20-18:28:59.321639 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56062: 129429 keys, 41699997 bytes, temperature: kWarm 2024/11/20-18:28:59.321704 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098539321673, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56062, "file_size": 41699997, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40101568, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131630190, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098537, "oldest_key_time": 0, "file_creation_time": 1732098537, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56062, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:28:59.373355 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056063.sst) /dev/shm/ToplingTemp/Topling-806427-EdJnWq 2024/11/20-18:29:00.059351 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056063.sst): first pass time = 0.69's, 195.667'MB/sec 2024/11/20-18:29:00.605333 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056063.sst): old prealloc_size = 147639500, real_size = 40350384 2024/11/20-18:29:00.605462 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056063.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.00 avg-zval = 309.81 2024/11/20-18:29:00.605497 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056063.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:29:00.768937 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056063.sst): second pass time = 0.55's, 241.397'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 254.880'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2008.573'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.272'MB/sec write SST data time = 0.04's, 937.437'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 30322.505'MB/sec zip my value time = 0.55's, unzip length = 0.132'GB zip my value throughput = 241.397'MB/sec zip pipeline throughput = 241.397'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.81 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.81 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2826 dict = 1.00 all = 3.2189 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 5.396816 GB avg = 1.017 KB (by entry num) total key len = 0.106132 GB avg = 0.020 KB total ukey len = 0.063679 GB avg = 0.012 KB total ukey num = 0.005306622 Billion total entry num = 0.005306622 Billion write speed all = 0.157772850 MB/sec (with seq num) write speed all = 0.156555697 MB/sec (without seq num) 2024/11/20-18:29:00.821617 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56063: 129429 keys, 41697524 bytes, temperature: kWarm 2024/11/20-18:29:00.821676 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098540821648, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56063, "file_size": 41697524, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40099040, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629926, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098539, "oldest_key_time": 0, "file_creation_time": 1732098539, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56063, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:29:00.857952 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056064.sst) /dev/shm/ToplingTemp/Topling-806427-SRHmhq 2024/11/20-18:29:01.509131 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056064.sst): first pass time = 0.65's, 206.125'MB/sec 2024/11/20-18:29:02.030777 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056064.sst): old prealloc_size = 147639500, real_size = 40352976 2024/11/20-18:29:02.030892 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056064.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.01 avg-zval = 309.84 2024/11/20-18:29:02.030931 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056064.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:29:02.191405 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056064.sst): second pass time = 0.52's, 252.663'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 259.637'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2380.090'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.279'MB/sec write SST data time = 0.04's, 935.900'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 43356.851'MB/sec zip my value time = 0.52's, unzip length = 0.132'GB zip my value throughput = 252.663'MB/sec zip pipeline throughput = 252.663'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.84 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.84 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2824 dict = 1.00 all = 3.2187 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 5.528446 GB avg = 1.017 KB (by entry num) total key len = 0.108721 GB avg = 0.020 KB total ukey len = 0.065233 GB avg = 0.012 KB total ukey num = 0.005436050 Billion total entry num = 0.005436050 Billion write speed all = 0.161614379 MB/sec (with seq num) write speed all = 0.160367591 MB/sec (without seq num) 2024/11/20-18:29:02.240937 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56064: 129428 keys, 41700084 bytes, temperature: kWarm 2024/11/20-18:29:02.241002 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098542240972, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56064, "file_size": 41700084, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40101632, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629775, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098540, "oldest_key_time": 0, "file_creation_time": 1732098540, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56064, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:29:02.278835 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056065.sst) /dev/shm/ToplingTemp/Topling-806427-ZMucMq 2024/11/20-18:29:02.920538 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056065.sst): first pass time = 0.64's, 209.173'MB/sec 2024/11/20-18:29:03.505112 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056065.sst): old prealloc_size = 147639500, real_size = 40349424 2024/11/20-18:29:03.505246 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056065.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129432 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.98 avg-zval = 309.80 2024/11/20-18:29:03.505293 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056065.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:29:03.674663 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056065.sst): second pass time = 0.58's, 225.475'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 245.947'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1889.391'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.262'MB/sec write SST data time = 0.05's, 906.133'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45609.841'MB/sec zip my value time = 0.58's, unzip length = 0.132'GB zip my value throughput = 225.475'MB/sec zip pipeline throughput = 225.475'MB/sec entries = 129432 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.98 avg-zval = 309.80 usrkeys = 129432 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.98 avg-zval = 309.80 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0981 value = 3.2827 dict = 1.00 all = 3.2189 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 5.660076 GB avg = 1.017 KB (by entry num) total key len = 0.111310 GB avg = 0.020 KB total ukey len = 0.066786 GB avg = 0.012 KB total ukey num = 0.005565482 Billion total entry num = 0.005565482 Billion write speed all = 0.165455314 MB/sec (with seq num) write speed all = 0.164178895 MB/sec (without seq num) 2024/11/20-18:29:03.725149 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56065: 129432 keys, 41697145 bytes, temperature: kWarm 2024/11/20-18:29:03.725215 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098543725184, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56065, "file_size": 41697145, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40098080, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588640, "raw_average_key_size": 20, "raw_value_size": 131629930, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098542, "oldest_key_time": 0, "file_creation_time": 1732098542, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56065, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:29:03.767252 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056066.sst) /dev/shm/ToplingTemp/Topling-806427-KhI4Gp 2024/11/20-18:29:04.317611 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056066.sst): first pass time = 0.55's, 243.892'MB/sec 2024/11/20-18:29:04.908852 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056066.sst): old prealloc_size = 147639500, real_size = 40350752 2024/11/20-18:29:04.908970 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056066.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.00 avg-zval = 309.82 2024/11/20-18:29:04.909001 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056066.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:29:05.079795 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056066.sst): second pass time = 0.59's, 222.878'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 243.961'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2202.920'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.270'MB/sec write SST data time = 0.05's, 817.324'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 25930.122'MB/sec zip my value time = 0.59's, unzip length = 0.132'GB zip my value throughput = 222.878'MB/sec zip pipeline throughput = 245.006'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.82 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.82 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2826 dict = 1.00 all = 3.2188 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 5.791705 GB avg = 1.017 KB (by entry num) total key len = 0.113898 GB avg = 0.020 KB total ukey len = 0.068339 GB avg = 0.012 KB total ukey num = 0.005694912 Billion total entry num = 0.005694912 Billion write speed all = 0.169296297 MB/sec (with seq num) write speed all = 0.167990246 MB/sec (without seq num) 2024/11/20-18:29:05.133582 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56066: 129430 keys, 41698236 bytes, temperature: kWarm 2024/11/20-18:29:05.133660 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098545133620, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56066, "file_size": 41698236, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40099408, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629827, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098543, "oldest_key_time": 0, "file_creation_time": 1732098543, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56066, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:29:05.173876 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056067.sst) /dev/shm/ToplingTemp/Topling-806427-EXKGgs 2024/11/20-18:29:05.823552 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056067.sst): first pass time = 0.65's, 206.605'MB/sec 2024/11/20-18:29:06.435870 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056067.sst): old prealloc_size = 147639500, real_size = 40351616 2024/11/20-18:29:06.435985 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056067.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.00 avg-zval = 309.82 2024/11/20-18:29:06.436066 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056067.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:29:06.602671 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056067.sst): second pass time = 0.61's, 215.202'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 250.540'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2347.056'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.273'MB/sec write SST data time = 0.05's, 866.152'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 52743.599'MB/sec zip my value time = 0.61's, unzip length = 0.132'GB zip my value throughput = 215.202'MB/sec zip pipeline throughput = 225.886'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.82 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1018.72 avg-zval = 309.82 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 31909 bytes = 67824 } seq expand size = 223363 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0418 GB } UnZip/Zip{ index = 7.0980 value = 3.2825 dict = 1.00 all = 3.2120 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3113 } ---------------------------- total value len = 5.923559 GB avg = 1.017 KB (by entry num) total key len = 0.116487 GB avg = 0.020 KB total ukey len = 0.069892 GB avg = 0.012 KB total ukey num = 0.005824342 Billion total entry num = 0.005824342 Billion write speed all = 0.173142789 MB/sec (with seq num) write speed all = 0.171807113 MB/sec (without seq num) 2024/11/20-18:29:06.655229 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56067: 129430 keys, 41787183 bytes, temperature: kWarm 2024/11/20-18:29:06.655297 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098546655264, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56067, "file_size": 41787183, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 938460617, "table_properties": {"data_size": 40100272, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629815, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098545, "oldest_key_time": 0, "file_creation_time": 1732098545, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56067, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:29:06.698868 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056068.sst) /dev/shm/ToplingTemp/Topling-806427-3zy8Wq 2024/11/20-18:29:07.306741 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056068.sst): first pass time = 0.61's, 220.817'MB/sec 2024/11/20-18:29:07.878703 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056068.sst): old prealloc_size = 147639500, real_size = 40354992 2024/11/20-18:29:07.878822 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056068.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129426 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.03 avg-zval = 309.86 2024/11/20-18:29:07.878857 140008517842496 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056068.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:29:08.070505 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056068.sst): second pass time = 0.57's, 230.391'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.19's, 217.458'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2205.095'MB/sec (index lex order gen) rebuild zvType time = 0.14's, 0.225'MB/sec write SST data time = 0.05's, 869.369'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 47819.198'MB/sec zip my value time = 0.57's, unzip length = 0.132'GB zip my value throughput = 230.391'MB/sec zip pipeline throughput = 230.391'MB/sec entries = 129426 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.03 avg-zval = 309.86 usrkeys = 129426 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.03 avg-zval = 309.86 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0978 value = 3.2822 dict = 1.00 all = 3.2185 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3107 } ---------------------------- total value len = 6.055189 GB avg = 1.017 KB (by entry num) total key len = 0.119075 GB avg = 0.020 KB total ukey len = 0.071445 GB avg = 0.012 KB total ukey num = 0.005953768 Billion total entry num = 0.005953768 Billion write speed all = 0.176982830 MB/sec (with seq num) write speed all = 0.175617531 MB/sec (without seq num) 2024/11/20-18:29:08.121088 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56068: 129426 keys, 41702296 bytes, temperature: kWarm 2024/11/20-18:29:08.121143 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098548121113, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56068, "file_size": 41702296, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 40103648, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588520, "raw_average_key_size": 20, "raw_value_size": 131630197, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129426, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732098546, "oldest_key_time": 0, "file_creation_time": 1732098546, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56068, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:29:08.164402 140008517842496 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056069.sst) /dev/shm/ToplingTemp/Topling-806427-444Zzo 2024/11/20-18:29:08.213022 140008517842496 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056069.sst): first pass time = 0.05's, 776.615'MB/sec 2024/11/20-18:29:08.474435 140008517842496 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056069.sst): old prealloc_size = 147639500, real_size = 11400216 2024/11/20-18:29:08.474479 140008517842496 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056069.sst): index type = UintIndex_AllOne , store type = DictZipBlobStore usrkeys = 36386 prefix = 4 raw-key = 0.0003 GB zip-key = 0.0000 GB avg-key = 8.00 avg-zkey = 0.00 raw-val = 0.0370 GB zip-val = 0.0114 GB avg-val =1017.02 avg-zval = 313.05 2024/11/20-18:29:08.522098 140008517842496 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00/056069.sst): second pass time = 0.26's, 141.944'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.05's, 248.239'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 8.612'MB/sec (index lex order gen) rebuild zvType time = 0.03's, 0.262'MB/sec write SST data time = 0.01's, 917.533'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.358'MB, throughput = 7567.140'MB/sec zip my value time = 0.26's, unzip length = 0.037'GB zip my value throughput = 141.944'MB/sec zip pipeline throughput = 141.944'MB/sec entries = 36386 avg-key = 20.00 avg-zkey = 0.01 avg-val = 1017.02 avg-zval = 313.05 usrkeys = 36386 avg-key = 12.00 avg-zkey = 0.01 avg-val = 1023.18 avg-zval = 313.05 TagRS{ kinds = RS_Key0_TagN bytes = 5800 } TagArray{ size = 32014 bytes = 68048 } seq expand size = 224098 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0004 GB value = 0.0370 GB dict = 0.36 MB all = 0.0377 GB } __ZipSize{ index = 0.0000 GB value = 0.0114 GB dict = 0.36 MB all = 0.0118 GB } UnZip/Zip{ index =2274.1250 value = 3.2487 dict = 1.00 all = 3.1913 } Zip/UnZip{ index = 0.0004 value = 0.3078 dict = 1.00 all = 0.3134 } ---------------------------- total value len = 6.092418 GB avg = 1.017 KB (by entry num) total key len = 0.119803 GB avg = 0.020 KB total ukey len = 0.071882 GB avg = 0.012 KB total ukey num = 0.005990154 Billion total entry num = 0.005990154 Billion write speed all = 0.178068550 MB/sec (with seq num) write speed all = 0.176694925 MB/sec (without seq num) 2024/11/20-18:29:08.545320 140008517842496 [compaction_job.cc:1948] [default] [JOB 4908] Generated table #56069: 36386 keys, 11823728 bytes, temperature: kWarm 2024/11/20-18:29:08.545374 140008517842496 EVENT_LOG_v1 {"time_micros": 1732098548545344, "cf_name": "default", "job": 4908, "event": "table_file_creation", "file_number": 56069, "file_size": 11823728, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 939593616, "table_properties": {"data_size": 11390752, "index_size": 192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 727720, "raw_average_key_size": 20, "raw_value_size": 37005268, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 36386, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-UintIndex_AllOne-DictZipBlobStore", "creation_time": 1732098548, "oldest_key_time": 0, "file_creation_time": 1732098548, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 56069, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:29:08.561772 140008517842496 [compaction_job.cc:703] job-04908: subcompact[0], size: 1.930196 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04908/att-00 [056023,056024,056025,056026,056027,056028,056029,056030,056031,056032,056033,056034,056035,056036,056037,056038,056039,056040,056041,056042,056043,056044,056045,056046,056047,056048,056049,056050,056051,056052,056053,056054,056055,056056,056057,056058,056059,056060,056061,056062,056063,056064,056065,056066,056067,056068,056069] 2024/11/20-18:29:09.023953 140008517842496 (Original Log Time 2024/11/20-18:27:59.129262) EVENT_LOG_v1 {"time_micros": 1732098479129217, "job": 4908, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [56017], "files_L3": [55958, 55959, 55960, 55961, 55963, 55964, 55965, 55966, 55967, 55968, 55970, 55971, 55972, 55973, 55974, 55975, 55976, 55977, 55978, 55979, 55980, 55981, 55982, 55983, 55984, 55985, 55986, 55987, 55988, 55989, 55990, 55991, 55992, 55993, 55995, 55996, 55997, 55998, 55999, 56000, 56001, 56002, 56003, 56004, 56005, 55692, 55686, 55664, 55668, 55674, 55681, 55675], "score": 0.9861, "input_data_size": 1953023832, "oldest_snapshot_seqno": -1} 2024/11/20-18:29:09.023958 140008517842496 (Original Log Time 2024/11/20-18:29:08.945651) [compaction_job.cc:2022] [default] [JOB 4908] Compacted 1@2 + 52@3 files to L3 => 1930196012 bytes 2024/11/20-18:29:09.023960 140008517842496 (Original Log Time 2024/11/20-18:29:09.023797) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 47 0 0 0] max score 0.07, MB/sec: 28.1 rd, 27.8 wr, level 3, files in(1, 52) out(47 +0 blob) MB in(31.3, 1831.2 +0.0 blob) out(1840.8 +0.0 blob), read-write-amplify(118.3) write-amplify(58.8) OK, records in: 5990182, records dropped: 28 output_compression: Snappy 2024/11/20-18:29:09.023966 140008517842496 (Original Log Time 2024/11/20-18:29:09.023914) EVENT_LOG_v1 {"time_micros": 1732098549023836, "cf": "default", "job": 4908, "event": "compaction_finished", "compaction_time_micros": 69432511, "compaction_time_cpu_micros": 17175735, "output_level": 3, "num_output_files": 47, "total_output_size": 1930196012, "num_input_records": 5990182, "num_output_records": 5990154, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 1519160059, "file_range_sync_nanos": 0, "file_fsync_nanos": 41854, "file_prepare_write_nanos": 94845, "lsm_state": [0, 0, 0, 47, 0, 0, 0]} 2024/11/20-18:29:09.185766 140008517842496 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732096383,"starts":1732098478,"executesMs":69940,"compactionJobId":4908,"attempt":0,"compactionInputRawBytes":6211800933,"compactionInputZipBytes":1953023832,"compactionOutputRawBytes":0,"compactionOutputZipBytes":1930196012,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-9cf37bb5a2bd25e224bc506ed6272ebb-a618ab9b4e1cf882-00"}}, response = 2024/11/20-18:29:09.185872 140008517842496 INFO dcompact_worker.cpp:1464: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-04908/att-00: olev 3, work 69.941 s, result 9.087 ms, install 78.337 ms, input{raw 5.785 GiB zip 1.819 GiB}