2024/11/21-00:34:26.888866 140008198919744 DEBG dcompact_worker.cpp:2219: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00: fork to child time = 0.009774 sec 2024/11/21-00:34:26.889382 140008198919744 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00 2024/11/21-00:34:26.896109 140008198919744 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00 2024/11/21-00:34:26.897939 140008198919744 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/21-00:34:26.898233 140008198919744 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/21-00:34:26.939251 140008198919744 [version_set.cc:6211] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-22842/att-00/MANIFEST-049091 2024/11/21-00:34:26.955572 140008198919744 [version_set.cc:6254] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-22842/att-00/MANIFEST-049091 succeeded,manifest_file_number is 49091, next_file_number is 89329, last_sequence is 1361742459, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 89186 2024/11/21-00:34:26.955579 140008198919744 [version_set.cc:6269] Column family [default] (ID 0), log number is 0 2024/11/21-00:34:26.955666 140008198919744 [version_set.cc:5724] Creating manifest 89329 2024/11/21-00:34:26.982109 140008198919744 DEBG dcompact_worker.cpp:1225: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/21-00:34:27.001207 140008198919744 [compaction_job.cc:2437] [default] [JOB 22842] Compacting 1@2 + 14@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/21-00:34:27.001221 140008198919744 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 17044 Base level 2, inputs: [89327(24MB)], [89265(37MB) 89266(36MB) 89267(36MB) 89268(35MB) 89269(36MB) 89270(37MB) 89271(35MB) 89272(40MB) 89273(37MB) 89274(38MB) 89275(38MB) 89276(38MB) 89277(37MB) 89285(37MB)] 2024/11/21-00:34:27.007644 140008198919744 [top_zip_table.cc:59] core git_version_hash_info_is:commit 81964c21ebbcc178d8e99dae0fbf61bded2729cc Author: rockeet Date: Mon Nov 11 21:28:46 2024 +0800 Makefile: fix gen git-version-*.cpp GIT_PATH_ARG = :!src/terark/fsa :!tools/fsa :!src/terark/zbs :!tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/21-00:34:27.007671 140008198919744 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit 5dfecb7d107196f044ff5bf78dc39d2e725b9d9b Author: leipeng Date: Thu Nov 7 15:22:45 2024 +0800 Makefile: fix for x86_64 CPU older than haswell GIT_PATH_ARG = src/terark/fsa tools/fsa g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/21-00:34:27.007687 140008198919744 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit d44dd5ff72707128fe3c916077b43d2093c01f7c Author: leipeng Date: Sun Oct 27 23:38:26 2024 +0800 Fix for msvc, when toplingdb on msvc GIT_PATH_ARG = src/terark/zbs tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/21-00:34:27.007694 140008198919744 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit 79614a7e7dc27362fc5151b0797eb7d83c674c7b Author: rockeet Date: Sun Nov 10 15:02:21 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/21-00:34:27.007721 140008198919744 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit c46b65a52d9c74aa8b87aeb8438620ff2dbc669c Author: rockeet Date: Sun Nov 10 15:02:42 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/21-00:34:27.563475 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089330.sst) /dev/shm/ToplingTemp/Topling-1190309-a5AdfN 2024/11/21-00:34:28.080244 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089330.sst): first pass time = 0.52's, 259.742'MB/sec 2024/11/21-00:34:29.377032 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089330.sst): old prealloc_size = 147639500, real_size = 34809360 2024/11/21-00:34:29.377661 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089330.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926921 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0328 GB avg-val = 116.80 avg-zval = 35.40 2024/11/21-00:34:29.379214 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089330.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:34:30.280089 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089330.sst): second pass time = 1.30's, 83.579'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 43.214'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2986.575'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.300'MB/sec write SST data time = 0.13's, 302.422'MB/sec dict compress time = 8375638008.17's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 29561.747'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.579'MB/sec zip pipeline throughput = 120.368'MB/sec entries = 926921 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.80 avg-zval = 35.40 usrkeys = 926921 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.80 avg-zval = 35.40 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926921 bytes = 3360096 } seq expand size = 6488447 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0328 GB dict = 1.08 MB all = 0.0390 GB } UnZip/Zip{ index = 10.5152 value = 3.2993 dict = 1.00 all = 3.4394 } Zip/UnZip{ index = 0.0951 value = 0.3031 dict = 1.00 all = 0.2907 } ---------------------------- total value len = 563.180485 GB avg = 0.331 KB (by entry num) total key len = 44.340360 GB avg = 0.026 KB total ukey len = 32.812680 GB avg = 0.014 KB total ukey num = 2.321941497 Billion total entry num = 1.701443519 Billion write speed all = 25.909357682 MB/sec (with seq num) write speed all = 25.328856663 MB/sec (without seq num) 2024/11/21-00:34:30.332518 140008198919744 [compaction_job.cc:1948] [default] [JOB 22842] Generated table #89330: 926921 keys, 39023636 bytes, temperature: kWarm 2024/11/21-00:34:30.332645 140008198919744 EVENT_LOG_v1 {"time_micros": 1732120470332584, "cf_name": "default", "job": 22842, "event": "table_file_creation", "file_number": 89330, "file_size": 39023636, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1069697601, "largest_seqno": 1361715378, "table_properties": {"data_size": 32814464, "index_size": 1763008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25953788, "raw_average_key_size": 28, "raw_value_size": 108264014, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926921, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732120467, "oldest_key_time": 0, "file_creation_time": 1732120467, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 89330, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:34:30.405930 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089331.sst) /dev/shm/ToplingTemp/Topling-1190309-eaNg3M 2024/11/21-00:34:31.325699 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089331.sst): first pass time = 0.92's, 145.934'MB/sec 2024/11/21-00:34:32.871012 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089331.sst): old prealloc_size = 147639500, real_size = 34481472 2024/11/21-00:34:32.871604 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089331.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926967 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0325 GB avg-val = 116.79 avg-zval = 35.05 2024/11/21-00:34:32.872466 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089331.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:34:33.848700 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089331.sst): second pass time = 1.54's, 70.090'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.98's, 39.581'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3074.830'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.282'MB/sec write SST data time = 0.15's, 250.708'MB/sec dict compress time = 8318521569.50's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31102.209'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.090'MB/sec zip pipeline throughput = 173.302'MB/sec entries = 926967 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.79 avg-zval = 35.05 usrkeys = 926967 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.79 avg-zval = 35.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926967 bytes = 3360272 } seq expand size = 6488769 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0387 GB } UnZip/Zip{ index = 10.5123 value = 3.3326 dict = 1.00 all = 3.4685 } Zip/UnZip{ index = 0.0951 value = 0.3001 dict = 1.00 all = 0.2883 } ---------------------------- total value len = 563.295236 GB avg = 0.331 KB (by entry num) total key len = 44.366315 GB avg = 0.026 KB total ukey len = 32.831220 GB avg = 0.014 KB total ukey num = 2.322868464 Billion total entry num = 1.702370486 Billion write speed all = 25.911414974 MB/sec (with seq num) write speed all = 25.330686073 MB/sec (without seq num) 2024/11/21-00:34:33.903814 140008198919744 [compaction_job.cc:1948] [default] [JOB 22842] Generated table #89331: 926967 keys, 38695786 bytes, temperature: kWarm 2024/11/21-00:34:33.903889 140008198919744 EVENT_LOG_v1 {"time_micros": 1732120473903845, "cf_name": "default", "job": 22842, "event": "table_file_creation", "file_number": 89331, "file_size": 38695786, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1069697639, "largest_seqno": 1361715211, "table_properties": {"data_size": 32485936, "index_size": 1763584, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25955076, "raw_average_key_size": 28, "raw_value_size": 108262774, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926967, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732120470, "oldest_key_time": 0, "file_creation_time": 1732120470, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 89331, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:34:33.942704 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089332.sst) /dev/shm/ToplingTemp/Topling-1190309-2kC2TL 2024/11/21-00:34:34.847501 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089332.sst): first pass time = 0.90's, 148.345'MB/sec 2024/11/21-00:34:36.146242 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089332.sst): old prealloc_size = 147639500, real_size = 33737424 2024/11/21-00:34:36.146849 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089332.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926788 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0317 GB avg-val = 116.82 avg-zval = 34.25 2024/11/21-00:34:36.147690 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089332.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:34:37.086934 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089332.sst): second pass time = 1.30's, 83.417'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 40.222'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3024.261'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.286'MB/sec write SST data time = 0.13's, 291.175'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33292.182'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.417'MB/sec zip pipeline throughput = 141.565'MB/sec entries = 926788 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.82 avg-zval = 34.25 usrkeys = 926788 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.82 avg-zval = 34.25 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926788 bytes = 3243776 } seq expand size = 6487516 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.5137 value = 3.4108 dict = 1.00 all = 3.5474 } Zip/UnZip{ index = 0.0951 value = 0.2932 dict = 1.00 all = 0.2819 } ---------------------------- total value len = 563.409991 GB avg = 0.331 KB (by entry num) total key len = 44.392265 GB avg = 0.026 KB total ukey len = 32.849755 GB avg = 0.014 KB total ukey num = 2.323795252 Billion total entry num = 1.703297274 Billion write speed all = 25.913836550 MB/sec (with seq num) write speed all = 25.332871716 MB/sec (without seq num) 2024/11/21-00:34:37.133744 140008198919744 [compaction_job.cc:1948] [default] [JOB 22842] Generated table #89332: 926788 keys, 37835642 bytes, temperature: kWarm 2024/11/21-00:34:37.133804 140008198919744 EVENT_LOG_v1 {"time_micros": 1732120477133774, "cf_name": "default", "job": 22842, "event": "table_file_creation", "file_number": 89332, "file_size": 37835642, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1224550093, "largest_seqno": 1361715358, "table_properties": {"data_size": 31742528, "index_size": 1763008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25950064, "raw_average_key_size": 28, "raw_value_size": 108267676, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926788, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732120473, "oldest_key_time": 0, "file_creation_time": 1732120473, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 89332, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:34:37.171126 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089333.sst) /dev/shm/ToplingTemp/Topling-1190309-bcCj0N 2024/11/21-00:34:38.014202 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089333.sst): first pass time = 0.84's, 159.206'MB/sec 2024/11/21-00:34:39.240742 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089333.sst): old prealloc_size = 147639500, real_size = 33686800 2024/11/21-00:34:39.241312 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089333.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923101 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0317 GB avg-val = 117.40 avg-zval = 34.34 2024/11/21-00:34:39.242132 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089333.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:34:40.151100 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089333.sst): second pass time = 1.23's, 88.408'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 41.498'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3203.129'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.301'MB/sec write SST data time = 0.14's, 266.000'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 21991.634'MB/sec zip my value time = 1.23's, unzip length = 0.108'GB zip my value throughput = 88.408'MB/sec zip pipeline throughput = 160.202'MB/sec entries = 923101 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.40 avg-zval = 34.34 usrkeys = 923101 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.40 avg-zval = 34.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923101 bytes = 3230864 } seq expand size = 6461707 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.5143 value = 3.4186 dict = 1.00 all = 3.5530 } Zip/UnZip{ index = 0.0951 value = 0.2925 dict = 1.00 all = 0.2815 } ---------------------------- total value len = 563.524824 GB avg = 0.331 KB (by entry num) total key len = 44.418112 GB avg = 0.026 KB total ukey len = 32.868217 GB avg = 0.014 KB total ukey num = 2.324718353 Billion total entry num = 1.704220375 Billion write speed all = 25.916448726 MB/sec (with seq num) write speed all = 25.335244967 MB/sec (without seq num) 2024/11/21-00:34:40.198412 140008198919744 [compaction_job.cc:1948] [default] [JOB 22842] Generated table #89333: 923101 keys, 37775730 bytes, temperature: kWarm 2024/11/21-00:34:40.198478 140008198919744 EVENT_LOG_v1 {"time_micros": 1732120480198447, "cf_name": "default", "job": 22842, "event": "table_file_creation", "file_number": 89333, "file_size": 37775730, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1156808633, "largest_seqno": 1361715254, "table_properties": {"data_size": 31699968, "index_size": 1755904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25846828, "raw_average_key_size": 28, "raw_value_size": 108370942, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923101, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732120477, "oldest_key_time": 0, "file_creation_time": 1732120477, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 89333, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:34:40.239961 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089334.sst) /dev/shm/ToplingTemp/Topling-1190309-cTY54L 2024/11/21-00:34:41.083485 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089334.sst): first pass time = 0.84's, 159.122'MB/sec 2024/11/21-00:34:42.736737 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089334.sst): old prealloc_size = 147639500, real_size = 34425680 2024/11/21-00:34:42.737312 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089334.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928461 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0324 GB avg-val = 116.56 avg-zval = 34.93 2024/11/21-00:34:42.738111 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089334.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:34:43.642169 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089334.sst): second pass time = 1.65's, 65.492'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 42.426'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3235.873'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.303'MB/sec write SST data time = 0.14's, 276.551'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 26420.298'MB/sec zip my value time = 1.65's, unzip length = 0.108'GB zip my value throughput = 65.492'MB/sec zip pipeline throughput = 165.961'MB/sec entries = 928461 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.56 avg-zval = 34.93 usrkeys = 928461 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.56 avg-zval = 34.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928461 bytes = 3133568 } seq expand size = 6499227 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0324 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.5151 value = 3.3373 dict = 1.00 all = 3.4941 } Zip/UnZip{ index = 0.0951 value = 0.2996 dict = 1.00 all = 0.2862 } ---------------------------- total value len = 563.639544 GB avg = 0.331 KB (by entry num) total key len = 44.444109 GB avg = 0.026 KB total ukey len = 32.886787 GB avg = 0.014 KB total ukey num = 2.325646814 Billion total entry num = 1.705148836 Billion write speed all = 25.918590151 MB/sec (with seq num) write speed all = 25.337156283 MB/sec (without seq num) 2024/11/21-00:34:43.689561 140008198919744 [compaction_job.cc:1948] [default] [JOB 22842] Generated table #89334: 928461 keys, 38412330 bytes, temperature: kWarm 2024/11/21-00:34:43.689625 140008198919744 EVENT_LOG_v1 {"time_micros": 1732120483689594, "cf_name": "default", "job": 22842, "event": "table_file_creation", "file_number": 89334, "file_size": 38412330, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1279613853, "largest_seqno": 1361715297, "table_properties": {"data_size": 32427456, "index_size": 1765952, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25996908, "raw_average_key_size": 28, "raw_value_size": 108220954, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928461, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732120480, "oldest_key_time": 0, "file_creation_time": 1732120480, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 89334, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:34:43.726960 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089335.sst) /dev/shm/ToplingTemp/Topling-1190309-dmkjuM 2024/11/21-00:34:44.852982 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089335.sst): first pass time = 1.13's, 119.201'MB/sec 2024/11/21-00:34:46.260121 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089335.sst): old prealloc_size = 147639500, real_size = 36539536 2024/11/21-00:34:46.260685 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089335.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928030 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.97 raw-val = 0.1082 GB zip-val = 0.0345 GB avg-val = 116.63 avg-zval = 37.16 2024/11/21-00:34:46.261537 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089335.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:34:47.143402 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089335.sst): second pass time = 1.41's, 76.966'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 45.419'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3419.026'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.313'MB/sec write SST data time = 0.14's, 284.777'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 23369.934'MB/sec zip my value time = 1.41's, unzip length = 0.108'GB zip my value throughput = 76.966'MB/sec zip pipeline throughput = 76.966'MB/sec entries = 928030 avg-key = 28.00 avg-zkey = 1.97 avg-val = 116.63 avg-zval = 37.16 usrkeys = 928030 avg-key = 20.00 avg-zkey = 1.97 avg-val = 121.99 avg-zval = 37.16 TagRS{ kinds = RS_Key0_TagN bytes = 145120 } TagArray{ size = 710623 bytes = 2576016 } seq expand size = 4974361 multi value expand size = 0 cnt WriteAppend = 119 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0401 GB } UnZip/Zip{ index = 10.1729 value = 3.1387 dict = 1.00 all = 3.3456 } Zip/UnZip{ index = 0.0983 value = 0.3186 dict = 1.00 all = 0.2989 } ---------------------------- total value len = 563.752752 GB avg = 0.330 KB (by entry num) total key len = 44.470094 GB avg = 0.026 KB total ukey len = 32.905347 GB avg = 0.014 KB total ukey num = 2.326574844 Billion total entry num = 1.706076866 Billion write speed all = 25.920654709 MB/sec (with seq num) write speed all = 25.338991199 MB/sec (without seq num) 2024/11/21-00:34:47.194126 140008198919744 [compaction_job.cc:1948] [default] [JOB 22842] Generated table #89335: 928030 keys, 40117600 bytes, temperature: kWarm 2024/11/21-00:34:47.194198 140008198919744 EVENT_LOG_v1 {"time_micros": 1732120487194165, "cf_name": "default", "job": 22842, "event": "table_file_creation", "file_number": 89335, "file_size": 40117600, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1361713420, "table_properties": {"data_size": 34482816, "index_size": 1824512, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25984840, "raw_average_key_size": 28, "raw_value_size": 108232916, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928030, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732120483, "oldest_key_time": 0, "file_creation_time": 1732120483, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 89335, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:34:47.232268 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089336.sst) /dev/shm/ToplingTemp/Topling-1190309-l77dDN 2024/11/21-00:34:48.028302 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089336.sst): first pass time = 0.80's, 168.616'MB/sec 2024/11/21-00:34:49.363800 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089336.sst): old prealloc_size = 147639500, real_size = 36522656 2024/11/21-00:34:49.364387 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089336.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926982 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.97 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.79 avg-zval = 37.18 2024/11/21-00:34:49.364511 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089336.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:34:50.064990 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089336.sst): second pass time = 1.33's, 81.110'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.70's, 53.522'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3201.844'MB/sec (index lex order gen) rebuild zvType time = 0.58's, 0.403'MB/sec write SST data time = 0.13's, 299.640'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 18611.283'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.110'MB/sec zip pipeline throughput = 81.110'MB/sec entries = 926982 avg-key = 28.00 avg-zkey = 1.97 avg-val = 116.79 avg-zval = 37.18 usrkeys = 926982 avg-key = 20.00 avg-zkey = 1.97 avg-val = 116.80 avg-zval = 37.18 TagRS{ kinds = RS_Key0_TagN bytes = 144960 } TagArray{ size = 871 bytes = 2848 } seq expand size = 6097 multi value expand size = 0 cnt WriteAppend = 38 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.1750 value = 3.1409 dict = 1.00 all = 3.5764 } Zip/UnZip{ index = 0.0983 value = 0.3184 dict = 1.00 all = 0.2796 } ---------------------------- total value len = 563.861020 GB avg = 0.330 KB (by entry num) total key len = 44.496049 GB avg = 0.026 KB total ukey len = 32.923887 GB avg = 0.014 KB total ukey num = 2.327501826 Billion total entry num = 1.707003848 Billion write speed all = 25.923147262 MB/sec (with seq num) write speed all = 25.341240163 MB/sec (without seq num) 2024/11/21-00:34:50.112036 140008198919744 [compaction_job.cc:1948] [default] [JOB 22842] Generated table #89336: 926982 keys, 37528321 bytes, temperature: kWarm 2024/11/21-00:34:50.112106 140008198919744 EVENT_LOG_v1 {"time_micros": 1732120490112075, "cf_name": "default", "job": 22842, "event": "table_file_creation", "file_number": 89336, "file_size": 37528321, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 972340122, "table_properties": {"data_size": 34468624, "index_size": 1822080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25955496, "raw_average_key_size": 28, "raw_value_size": 108262300, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926982, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732120487, "oldest_key_time": 0, "file_creation_time": 1732120487, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 89336, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:34:50.151810 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089337.sst) /dev/shm/ToplingTemp/Topling-1190309-32YkXM 2024/11/21-00:34:51.070909 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089337.sst): first pass time = 0.92's, 146.037'MB/sec 2024/11/21-00:34:52.409303 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089337.sst): old prealloc_size = 147639500, real_size = 37840720 2024/11/21-00:34:52.410008 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089337.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927153 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1083 GB zip-val = 0.0358 GB avg-val = 116.76 avg-zval = 38.62 2024/11/21-00:34:52.410828 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089337.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:34:53.242029 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089337.sst): second pass time = 1.34's, 80.941'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 50.107'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2646.657'MB/sec (index lex order gen) rebuild zvType time = 0.67's, 0.345'MB/sec write SST data time = 0.16's, 259.783'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 21908.802'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.941'MB/sec zip pipeline throughput = 80.941'MB/sec entries = 927153 avg-key = 28.00 avg-zkey = 1.94 avg-val = 116.76 avg-zval = 38.62 usrkeys = 927153 avg-key = 20.00 avg-zkey = 1.94 avg-val = 122.76 avg-zval = 38.62 TagRS{ kinds = RS_Key0_TagN bytes = 144960 } TagArray{ size = 794588 bytes = 2880400 } seq expand size = 5562116 multi value expand size = 0 cnt WriteAppend = 25 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0358 GB dict = 1.08 MB all = 0.0417 GB } UnZip/Zip{ index = 10.3105 value = 3.0231 dict = 1.00 all = 3.2168 } Zip/UnZip{ index = 0.0970 value = 0.3308 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 563.974840 GB avg = 0.330 KB (by entry num) total key len = 44.522010 GB avg = 0.026 KB total ukey len = 32.942430 GB avg = 0.014 KB total ukey num = 2.328428979 Billion total entry num = 1.707931001 Billion write speed all = 25.925593753 MB/sec (with seq num) write speed all = 25.343449403 MB/sec (without seq num) 2024/11/21-00:34:53.293308 140008198919744 [compaction_job.cc:1948] [default] [JOB 22842] Generated table #89337: 927153 keys, 41724420 bytes, temperature: kWarm 2024/11/21-00:34:53.293378 140008198919744 EVENT_LOG_v1 {"time_micros": 1732120493293346, "cf_name": "default", "job": 22842, "event": "table_file_creation", "file_number": 89337, "file_size": 41724420, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1323177458, "table_properties": {"data_size": 35810304, "index_size": 1798464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25960284, "raw_average_key_size": 28, "raw_value_size": 108257550, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927153, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732120490, "oldest_key_time": 0, "file_creation_time": 1732120490, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 89337, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:34:53.329061 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089338.sst) /dev/shm/ToplingTemp/Topling-1190309-bK3FwO 2024/11/21-00:34:54.254375 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089338.sst): first pass time = 0.93's, 145.058'MB/sec 2024/11/21-00:34:55.557955 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089338.sst): old prealloc_size = 147639500, real_size = 35440768 2024/11/21-00:34:55.558615 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089338.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926100 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0335 GB avg-val = 116.93 avg-zval = 36.13 2024/11/21-00:34:55.559458 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089338.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:34:56.394449 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089338.sst): second pass time = 1.30's, 83.124'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 47.409'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2759.463'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 314.534'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33713.403'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.124'MB/sec zip pipeline throughput = 83.124'MB/sec entries = 926100 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.93 avg-zval = 36.13 usrkeys = 926100 avg-key = 20.00 avg-zkey = 1.89 avg-val = 123.93 avg-zval = 36.13 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926100 bytes = 3357120 } seq expand size = 6482700 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0335 GB dict = 1.08 MB all = 0.0397 GB } UnZip/Zip{ index = 10.5843 value = 3.2364 dict = 1.00 all = 3.3845 } Zip/UnZip{ index = 0.0945 value = 0.3090 dict = 1.00 all = 0.2955 } ---------------------------- total value len = 564.089609 GB avg = 0.330 KB (by entry num) total key len = 44.547940 GB avg = 0.026 KB total ukey len = 32.960952 GB avg = 0.014 KB total ukey num = 2.329355079 Billion total entry num = 1.708857101 Billion write speed all = 25.928105980 MB/sec (with seq num) write speed all = 25.345724193 MB/sec (without seq num) 2024/11/21-00:34:56.445551 140008198919744 [compaction_job.cc:1948] [default] [JOB 22842] Generated table #89338: 926100 keys, 39656770 bytes, temperature: kWarm 2024/11/21-00:34:56.445620 140008198919744 EVENT_LOG_v1 {"time_micros": 1732120496445588, "cf_name": "default", "job": 22842, "event": "table_file_creation", "file_number": 89338, "file_size": 39656770, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1057126592, "largest_seqno": 1331715475, "table_properties": {"data_size": 33459120, "index_size": 1749952, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25930800, "raw_average_key_size": 28, "raw_value_size": 108287048, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926100, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732120493, "oldest_key_time": 0, "file_creation_time": 1732120493, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 89338, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:34:56.481701 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089339.sst) /dev/shm/ToplingTemp/Topling-1190309-muVRnO 2024/11/21-00:34:57.371726 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089339.sst): first pass time = 0.89's, 150.811'MB/sec 2024/11/21-00:34:58.577205 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089339.sst): old prealloc_size = 147639500, real_size = 35443024 2024/11/21-00:34:58.577823 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089339.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927917 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0334 GB avg-val = 116.64 avg-zval = 36.04 2024/11/21-00:34:58.578652 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089339.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:34:59.409697 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089339.sst): second pass time = 1.20's, 89.850'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.501'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3060.118'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.329'MB/sec write SST data time = 0.13's, 311.084'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34249.580'MB/sec zip my value time = 1.20's, unzip length = 0.108'GB zip my value throughput = 89.850'MB/sec zip pipeline throughput = 89.850'MB/sec entries = 927917 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.64 avg-zval = 36.04 usrkeys = 927917 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.64 avg-zval = 36.04 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927917 bytes = 3247728 } seq expand size = 6495419 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0334 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.5124 value = 3.2362 dict = 1.00 all = 3.3941 } Zip/UnZip{ index = 0.0951 value = 0.3090 dict = 1.00 all = 0.2946 } ---------------------------- total value len = 564.204341 GB avg = 0.330 KB (by entry num) total key len = 44.573922 GB avg = 0.026 KB total ukey len = 32.979510 GB avg = 0.014 KB total ukey num = 2.330282996 Billion total entry num = 1.709785018 Billion write speed all = 25.930769582 MB/sec (with seq num) write speed all = 25.348146397 MB/sec (without seq num) 2024/11/21-00:34:59.457416 140008198919744 [compaction_job.cc:1948] [default] [JOB 22842] Generated table #89339: 927917 keys, 39544584 bytes, temperature: kWarm 2024/11/21-00:34:59.457481 140008198919744 EVENT_LOG_v1 {"time_micros": 1732120499457450, "cf_name": "default", "job": 22842, "event": "table_file_creation", "file_number": 89339, "file_size": 39544584, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1187855354, "largest_seqno": 1331717282, "table_properties": {"data_size": 33445504, "index_size": 1765376, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25981676, "raw_average_key_size": 28, "raw_value_size": 108236130, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927917, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732120496, "oldest_key_time": 0, "file_creation_time": 1732120496, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 89339, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:34:59.493401 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089340.sst) /dev/shm/ToplingTemp/Topling-1190309-Y5WZbO 2024/11/21-00:35:00.314355 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089340.sst): first pass time = 0.82's, 163.499'MB/sec 2024/11/21-00:35:01.484837 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089340.sst): old prealloc_size = 147639500, real_size = 35749056 2024/11/21-00:35:01.485483 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089340.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927571 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0337 GB avg-val = 116.70 avg-zval = 36.38 2024/11/21-00:35:01.486314 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089340.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:35:02.519698 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089340.sst): second pass time = 1.17's, 92.550'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 38.511'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2922.055'MB/sec (index lex order gen) rebuild zvType time = 0.89's, 0.261'MB/sec write SST data time = 0.15's, 271.217'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 19287.589'MB/sec zip my value time = 1.17's, unzip length = 0.108'GB zip my value throughput = 92.550'MB/sec zip pipeline throughput = 92.550'MB/sec entries = 927571 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.70 avg-zval = 36.38 usrkeys = 927571 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.70 avg-zval = 36.38 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927571 bytes = 3246512 } seq expand size = 6492997 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0337 GB dict = 1.08 MB all = 0.0399 GB } UnZip/Zip{ index = 10.4437 value = 3.2082 dict = 1.00 all = 3.3678 } Zip/UnZip{ index = 0.0958 value = 0.3117 dict = 1.00 all = 0.2969 } ---------------------------- total value len = 564.319080 GB avg = 0.330 KB (by entry num) total key len = 44.599894 GB avg = 0.026 KB total ukey len = 32.998062 GB avg = 0.014 KB total ukey num = 2.331210567 Billion total entry num = 1.710712589 Billion write speed all = 25.933327745 MB/sec (with seq num) write speed all = 25.350465694 MB/sec (without seq num) 2024/11/21-00:35:02.569773 140008198919744 [compaction_job.cc:1948] [default] [JOB 22842] Generated table #89340: 927571 keys, 39853582 bytes, temperature: kWarm 2024/11/21-00:35:02.569846 140008198919744 EVENT_LOG_v1 {"time_micros": 1732120502569812, "cf_name": "default", "job": 22842, "event": "table_file_creation", "file_number": 89340, "file_size": 39853582, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1187855426, "largest_seqno": 1331716806, "table_properties": {"data_size": 33740656, "index_size": 1776320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25971988, "raw_average_key_size": 28, "raw_value_size": 108245802, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927571, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732120499, "oldest_key_time": 0, "file_creation_time": 1732120499, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 89340, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:35:02.609179 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089341.sst) /dev/shm/ToplingTemp/Topling-1190309-JzXTrK 2024/11/21-00:35:03.677377 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089341.sst): first pass time = 1.07's, 125.654'MB/sec 2024/11/21-00:35:05.019189 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089341.sst): old prealloc_size = 147639500, real_size = 36385328 2024/11/21-00:35:05.019859 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089341.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929942 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0344 GB avg-val = 116.33 avg-zval = 36.96 2024/11/21-00:35:05.020692 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089341.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:35:05.836350 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089341.sst): second pass time = 1.34's, 80.671'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 49.553'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2813.799'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.12's, 326.892'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 19858.927'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.671'MB/sec zip pipeline throughput = 80.671'MB/sec entries = 929942 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.33 avg-zval = 36.96 usrkeys = 929942 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.33 avg-zval = 36.96 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929942 bytes = 3254816 } seq expand size = 6509594 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0405 GB } UnZip/Zip{ index = 10.4456 value = 3.1473 dict = 1.00 all = 3.3146 } Zip/UnZip{ index = 0.0957 value = 0.3177 dict = 1.00 all = 0.3017 } ---------------------------- total value len = 564.433769 GB avg = 0.330 KB (by entry num) total key len = 44.625932 GB avg = 0.026 KB total ukey len = 33.016660 GB avg = 0.014 KB total ukey num = 2.332140509 Billion total entry num = 1.711642531 Billion write speed all = 25.935657703 MB/sec (with seq num) write speed all = 25.352561173 MB/sec (without seq num) 2024/11/21-00:35:05.887579 140008198919744 [compaction_job.cc:1948] [default] [JOB 22842] Generated table #89341: 929942 keys, 40492952 bytes, temperature: kWarm 2024/11/21-00:35:05.887654 140008198919744 EVENT_LOG_v1 {"time_micros": 1732120505887615, "cf_name": "default", "job": 22842, "event": "table_file_creation", "file_number": 89341, "file_size": 40492952, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1187851486, "largest_seqno": 1331717093, "table_properties": {"data_size": 34372128, "index_size": 1780544, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26038376, "raw_average_key_size": 28, "raw_value_size": 108179444, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929942, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732120502, "oldest_key_time": 0, "file_creation_time": 1732120502, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 89341, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:35:05.928433 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089342.sst) /dev/shm/ToplingTemp/Topling-1190309-8IpvxO 2024/11/21-00:35:06.894483 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089342.sst): first pass time = 0.97's, 138.941'MB/sec 2024/11/21-00:35:08.247504 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089342.sst): old prealloc_size = 147639500, real_size = 35356144 2024/11/21-00:35:08.248127 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089342.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927357 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0334 GB avg-val = 116.73 avg-zval = 35.97 2024/11/21-00:35:08.248961 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089342.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:35:09.205713 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089342.sst): second pass time = 1.35's, 80.055'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 41.394'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2913.844'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.284'MB/sec write SST data time = 0.14's, 280.335'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 24656.565'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.055'MB/sec zip pipeline throughput = 171.406'MB/sec entries = 927357 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.73 avg-zval = 35.97 usrkeys = 927357 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.61 avg-zval = 35.97 TagRS{ kinds = RS_Key0_TagN bytes = 145000 } TagArray{ size = 911326 bytes = 3303568 } seq expand size = 6379282 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0334 GB dict = 1.08 MB all = 0.0397 GB } UnZip/Zip{ index = 10.5148 value = 3.2449 dict = 1.00 all = 3.3840 } Zip/UnZip{ index = 0.0951 value = 0.3082 dict = 1.00 all = 0.2955 } ---------------------------- total value len = 564.548400 GB avg = 0.330 KB (by entry num) total key len = 44.651898 GB avg = 0.026 KB total ukey len = 33.035208 GB avg = 0.014 KB total ukey num = 2.333067866 Billion total entry num = 1.712569888 Billion write speed all = 25.937923241 MB/sec (with seq num) write speed all = 25.354594488 MB/sec (without seq num) 2024/11/21-00:35:09.255938 140008198919744 [compaction_job.cc:1948] [default] [JOB 22842] Generated table #89342: 927357 keys, 39662906 bytes, temperature: kWarm 2024/11/21-00:35:09.256009 140008198919744 EVENT_LOG_v1 {"time_micros": 1732120509255975, "cf_name": "default", "job": 22842, "event": "table_file_creation", "file_number": 89342, "file_size": 39662906, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1361715414, "table_properties": {"data_size": 33360224, "index_size": 1763904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25965996, "raw_average_key_size": 28, "raw_value_size": 108251758, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927357, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732120505, "oldest_key_time": 0, "file_creation_time": 1732120505, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 89342, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:35:09.290958 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089343.sst) /dev/shm/ToplingTemp/Topling-1190309-Xa3OIO 2024/11/21-00:35:10.149007 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089343.sst): first pass time = 0.86's, 156.429'MB/sec 2024/11/21-00:35:11.321277 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089343.sst): old prealloc_size = 147639500, real_size = 34561936 2024/11/21-00:35:11.321842 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089343.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930086 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0326 GB avg-val = 116.31 avg-zval = 35.01 2024/11/21-00:35:11.322651 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089343.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-00:35:12.163745 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089343.sst): second pass time = 1.17's, 92.341'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 45.764'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3244.967'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 296.688'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 27199.919'MB/sec zip my value time = 1.17's, unzip length = 0.108'GB zip my value throughput = 92.341'MB/sec zip pipeline throughput = 92.341'MB/sec entries = 930086 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.31 avg-zval = 35.01 usrkeys = 930086 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.31 avg-zval = 35.01 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930086 bytes = 3139056 } seq expand size = 6510602 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0326 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.5133 value = 3.3224 dict = 1.00 all = 3.4814 } Zip/UnZip{ index = 0.0951 value = 0.3010 dict = 1.00 all = 0.2872 } ---------------------------- total value len = 564.663086 GB avg = 0.330 KB (by entry num) total key len = 44.677941 GB avg = 0.026 KB total ukey len = 33.053809 GB avg = 0.014 KB total ukey num = 2.333997952 Billion total entry num = 1.713499974 Billion write speed all = 25.940647961 MB/sec (with seq num) write speed all = 25.357075903 MB/sec (without seq num) 2024/11/21-00:35:12.209653 140008198919744 [compaction_job.cc:1948] [default] [JOB 22842] Generated table #89343: 930086 keys, 38553304 bytes, temperature: kWarm 2024/11/21-00:35:12.209712 140008198919744 EVENT_LOG_v1 {"time_micros": 1732120512209681, "cf_name": "default", "job": 22842, "event": "table_file_creation", "file_number": 89343, "file_size": 38553304, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1270579075, "largest_seqno": 1361714980, "table_properties": {"data_size": 32559872, "index_size": 1769344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26042408, "raw_average_key_size": 28, "raw_value_size": 108175324, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930086, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732120509, "oldest_key_time": 0, "file_creation_time": 1732120509, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 89343, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:35:12.243482 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089344.sst) /dev/shm/ToplingTemp/Topling-1190309-dfhPAL 2024/11/21-00:35:12.311723 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089344.sst): first pass time = 0.07's, 248.809'MB/sec 2024/11/21-00:35:12.449497 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089344.sst): old prealloc_size = 147639500, real_size = 5098896 2024/11/21-00:35:12.449637 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089344.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 116326 prefix = 4 raw-key = 0.0019 GB zip-key = 0.0002 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0137 GB zip-val = 0.0048 GB avg-val = 117.90 avg-zval = 41.67 2024/11/21-00:35:12.449779 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089344.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0002 GB, waited 0.000 sec, Key+Value = 0.017 GB 2024/11/21-00:35:12.552621 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00/089344.sst): second pass time = 0.14's, 100.030'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.10's, 54.669'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1859.214'MB/sec (index lex order gen) rebuild zvType time = 0.09's, 0.337'MB/sec write SST data time = 0.02's, 336.482'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.143'MB, throughput = 6004.017'MB/sec zip my value time = 0.14's, unzip length = 0.014'GB zip my value throughput = 100.030'MB/sec zip pipeline throughput = 100.030'MB/sec entries = 116326 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.90 avg-zval = 41.67 usrkeys = 116326 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.90 avg-zval = 41.67 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 116326 bytes = 421696 } seq expand size = 814282 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0023 GB value = 0.0137 GB dict = 0.14 MB all = 0.0170 GB } __ZipSize{ index = 0.0002 GB value = 0.0048 GB dict = 0.14 MB all = 0.0056 GB } UnZip/Zip{ index = 10.4429 value = 2.8297 dict = 1.00 all = 3.0109 } Zip/UnZip{ index = 0.0958 value = 0.3534 dict = 1.00 all = 0.3321 } ---------------------------- total value len = 564.677615 GB avg = 0.330 KB (by entry num) total key len = 44.681198 GB avg = 0.026 KB total ukey len = 33.056136 GB avg = 0.014 KB total ukey num = 2.334114278 Billion total entry num = 1.713616300 Billion write speed all = 25.940975693 MB/sec (with seq num) write speed all = 25.357373679 MB/sec (without seq num) 2024/11/21-00:35:12.566838 140008198919744 [compaction_job.cc:1948] [default] [JOB 22842] Generated table #89344: 116326 keys, 5636805 bytes, temperature: kWarm 2024/11/21-00:35:12.566888 140008198919744 EVENT_LOG_v1 {"time_micros": 1732120512566861, "cf_name": "default", "job": 22842, "event": "table_file_creation", "file_number": 89344, "file_size": 5636805, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1053131138, "largest_seqno": 1360210149, "table_properties": {"data_size": 4846848, "index_size": 222784, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3257128, "raw_average_key_size": 28, "raw_value_size": 13714944, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 116326, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732120512, "oldest_key_time": 0, "file_creation_time": 1732120512, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 89344, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-00:35:12.576854 140008198919744 [compaction_job.cc:703] job-22842: subcompact[0], size: 0.554514 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22842/att-00 [089330,089331,089332,089333,089334,089335,089336,089337,089338,089339,089340,089341,089342,089343,089344] 2024/11/21-00:35:12.696809 140008198919744 (Original Log Time 2024/11/21-00:34:27.001272) EVENT_LOG_v1 {"time_micros": 1732120467001234, "job": 22842, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [89327], "files_L3": [89265, 89266, 89267, 89268, 89269, 89270, 89271, 89272, 89273, 89274, 89275, 89276, 89277, 89285], "score": 0.992974, "input_data_size": 575056647, "oldest_snapshot_seqno": 1361742449} 2024/11/21-00:35:12.696813 140008198919744 (Original Log Time 2024/11/21-00:35:12.692137) [compaction_job.cc:2022] [default] [JOB 22842] Compacted 1@2 + 14@3 files to L3 => 554514368 bytes 2024/11/21-00:35:12.696814 140008198919744 (Original Log Time 2024/11/21-00:35:12.696743) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 12.6 rd, 12.2 wr, level 3, files in(1, 14) out(15 +0 blob) MB in(24.4, 524.0 +0.0 blob) out(528.8 +0.0 blob), read-write-amplify(44.2) write-amplify(21.7) OK, records in: 13192638, records dropped: 92936 output_compression: Snappy 2024/11/21-00:35:12.696818 140008198919744 (Original Log Time 2024/11/21-00:35:12.696788) EVENT_LOG_v1 {"time_micros": 1732120512696764, "cf": "default", "job": 22842, "event": "compaction_finished", "compaction_time_micros": 45575586, "compaction_time_cpu_micros": 20336820, "output_level": 3, "num_output_files": 15, "total_output_size": 554514368, "num_input_records": 13192638, "num_output_records": 13099702, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 410949137, "file_range_sync_nanos": 0, "file_fsync_nanos": 14016, "file_prepare_write_nanos": 47871, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/21-00:35:12.829123 140008198919744 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732096383,"starts":1732120466,"executesMs":45797,"compactionJobId":22842,"attempt":0,"compactionInputRawBytes":1909218144,"compactionInputZipBytes":575056647,"compactionOutputRawBytes":0,"compactionOutputZipBytes":554514368,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-f021080a271f56722cdd295f80e84cd6-9d4c886fed75ddc3-00"}}, response = 2024/11/21-00:35:12.829175 140008198919744 INFO dcompact_worker.cpp:1464: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-22842/att-00: olev 3, work 45.797 s, result 5.919 ms, install 4.696 ms, input{raw 1.778 GiB zip 548.417 MiB}