2024/11/20-19:33:03.720372 140008266061376 DEBG dcompact_worker.cpp:2219: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00: fork to child time = 0.009641 sec 2024/11/20-19:33:03.720951 140008266061376 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00 2024/11/20-19:33:03.727696 140008266061376 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00 2024/11/20-19:33:03.729548 140008266061376 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/20-19:33:03.729773 140008266061376 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/20-19:33:03.789443 140008266061376 [version_set.cc:6211] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-08284/att-00/MANIFEST-049091 2024/11/20-19:33:03.857582 140008266061376 [version_set.cc:6254] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-08284/att-00/MANIFEST-049091 succeeded,manifest_file_number is 49091, next_file_number is 62173, last_sequence is 1024773869, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 62005 2024/11/20-19:33:03.857597 140008266061376 [version_set.cc:6269] Column family [default] (ID 0), log number is 0 2024/11/20-19:33:03.857686 140008266061376 [version_set.cc:5724] Creating manifest 62173 2024/11/20-19:33:03.890956 140008266061376 DEBG dcompact_worker.cpp:1225: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/20-19:33:03.909165 140008266061376 [compaction_job.cc:2437] [default] [JOB 8284] Compacting 1@2 + 11@3 files to L3, score 0.81, subcompactions 1 : 1 2024/11/20-19:33:03.909181 140008266061376 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 7413 Base level 2, inputs: [62141(27MB)], [62110(36MB) 62111(37MB) 62112(38MB) 62113(38MB) 62114(15MB) 62056(37MB) 62057(36MB) 62058(36MB) 62059(35MB) 62060(37MB) 62161(38MB)] 2024/11/20-19:33:03.915923 140008266061376 [top_zip_table.cc:59] core git_version_hash_info_is:commit 81964c21ebbcc178d8e99dae0fbf61bded2729cc Author: rockeet Date: Mon Nov 11 21:28:46 2024 +0800 Makefile: fix gen git-version-*.cpp GIT_PATH_ARG = :!src/terark/fsa :!tools/fsa :!src/terark/zbs :!tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-19:33:03.915957 140008266061376 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit 5dfecb7d107196f044ff5bf78dc39d2e725b9d9b Author: leipeng Date: Thu Nov 7 15:22:45 2024 +0800 Makefile: fix for x86_64 CPU older than haswell GIT_PATH_ARG = src/terark/fsa tools/fsa g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-19:33:03.915977 140008266061376 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit d44dd5ff72707128fe3c916077b43d2093c01f7c Author: leipeng Date: Sun Oct 27 23:38:26 2024 +0800 Fix for msvc, when toplingdb on msvc GIT_PATH_ARG = src/terark/zbs tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-19:33:03.915985 140008266061376 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit 79614a7e7dc27362fc5151b0797eb7d83c674c7b Author: rockeet Date: Sun Nov 10 15:02:21 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-19:33:03.915994 140008266061376 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit c46b65a52d9c74aa8b87aeb8438620ff2dbc669c Author: rockeet Date: Sun Nov 10 15:02:42 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-19:33:04.466023 140008266061376 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062174.sst) /dev/shm/ToplingTemp/Topling-891171-ixZOQk 2024/11/20-19:33:05.270075 140008266061376 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062174.sst): first pass time = 0.80's, 166.939'MB/sec 2024/11/20-19:33:06.816617 140008266061376 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062174.sst): old prealloc_size = 147639500, real_size = 36629488 2024/11/20-19:33:06.817161 140008266061376 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062174.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927330 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0346 GB avg-val = 116.74 avg-zval = 37.32 2024/11/20-19:33:06.817636 140008266061376 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062174.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-19:33:07.664919 140008266061376 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062174.sst): second pass time = 1.55's, 70.039'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.547'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3418.336'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.15's, 266.430'MB/sec dict compress time = 2320536028.68's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 22510.638'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 70.039'MB/sec zip pipeline throughput = 70.039'MB/sec entries = 927330 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.74 avg-zval = 37.32 usrkeys = 927330 avg-key = 20.00 avg-zkey = 1.93 avg-val = 118.98 avg-zval = 37.32 TagRS{ kinds = RS_Key0_TagN bytes = 145000 } TagArray{ size = 297669 bytes = 1004640 } seq expand size = 2083683 multi value expand size = 0 cnt WriteAppend = 28 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0346 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.3767 value = 3.1278 dict = 1.00 all = 3.4738 } Zip/UnZip{ index = 0.0964 value = 0.3197 dict = 1.00 all = 0.2879 } ---------------------------- total value len = 0.110336 GB avg = 0.119 KB (by entry num) total key len = 0.025965 GB avg = 0.028 KB total ukey len = 0.018547 GB avg = 0.020 KB total ukey num = 0.000927330 Billion total entry num = 0.000927330 Billion write speed all = 0.003519652 MB/sec (with seq num) write speed all = 0.003328084 MB/sec (without seq num) 2024/11/20-19:33:07.713354 140008266061376 [compaction_job.cc:1948] [default] [JOB 8284] Generated table #62174: 927330 keys, 38637033 bytes, temperature: kWarm 2024/11/20-19:33:07.713449 140008266061376 EVENT_LOG_v1 {"time_micros": 1732102387713408, "cf_name": "default", "job": 8284, "event": "table_file_creation", "file_number": 62174, "file_size": 38637033, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1024548829, "table_properties": {"data_size": 34610144, "index_size": 1787328, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25965240, "raw_average_key_size": 28, "raw_value_size": 108252512, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927330, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732102384, "oldest_key_time": 0, "file_creation_time": 1732102384, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 62174, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:33:07.750596 140008266061376 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062175.sst) /dev/shm/ToplingTemp/Topling-891171-tZZ0Mk 2024/11/20-19:33:08.950639 140008266061376 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062175.sst): first pass time = 1.20's, 111.847'MB/sec 2024/11/20-19:33:10.220251 140008266061376 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062175.sst): old prealloc_size = 147639500, real_size = 36553824 2024/11/20-19:33:10.220827 140008266061376 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062175.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923944 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0346 GB avg-val = 117.27 avg-zval = 37.42 2024/11/20-19:33:10.221309 140008266061376 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062175.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-19:33:10.986023 140008266061376 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062175.sst): second pass time = 1.27's, 85.392'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.77's, 51.108'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3171.003'MB/sec (index lex order gen) rebuild zvType time = 0.64's, 0.363'MB/sec write SST data time = 0.13's, 305.041'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 24158.720'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 85.392'MB/sec zip pipeline throughput = 85.392'MB/sec entries = 923944 avg-key = 28.00 avg-zkey = 1.89 avg-val = 117.27 avg-zval = 37.42 usrkeys = 923944 avg-key = 20.00 avg-zkey = 1.89 avg-val = 120.94 avg-zval = 37.42 TagRS{ kinds = RS_Key0_TagN bytes = 144480 } TagArray{ size = 485518 bytes = 1577952 } seq expand size = 3398626 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0346 GB dict = 1.08 MB all = 0.0391 GB } UnZip/Zip{ index = 10.5825 value = 3.1336 dict = 1.00 all = 3.4294 } Zip/UnZip{ index = 0.0945 value = 0.3191 dict = 1.00 all = 0.2916 } ---------------------------- total value len = 0.222082 GB avg = 0.120 KB (by entry num) total key len = 0.051836 GB avg = 0.028 KB total ukey len = 0.037025 GB avg = 0.020 KB total ukey num = 0.001851274 Billion total entry num = 0.001851274 Billion write speed all = 0.007072656 MB/sec (with seq num) write speed all = 0.006690252 MB/sec (without seq num) 2024/11/20-19:33:11.043349 140008266061376 [compaction_job.cc:1948] [default] [JOB 8284] Generated table #62175: 923944 keys, 39137013 bytes, temperature: kWarm 2024/11/20-19:33:11.043441 140008266061376 EVENT_LOG_v1 {"time_micros": 1732102391043396, "cf_name": "default", "job": 8284, "event": "table_file_creation", "file_number": 62175, "file_size": 39137013, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1024548462, "table_properties": {"data_size": 34576464, "index_size": 1746176, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25870432, "raw_average_key_size": 28, "raw_value_size": 108347436, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923944, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732102387, "oldest_key_time": 0, "file_creation_time": 1732102387, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 62175, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:33:11.097995 140008266061376 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062176.sst) /dev/shm/ToplingTemp/Topling-891171-LJ0nRm 2024/11/20-19:33:12.177252 140008266061376 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062176.sst): first pass time = 1.08's, 124.371'MB/sec 2024/11/20-19:33:13.505350 140008266061376 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062176.sst): old prealloc_size = 147639500, real_size = 36308176 2024/11/20-19:33:13.506023 140008266061376 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062176.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923028 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0343 GB avg-val = 117.41 avg-zval = 37.18 2024/11/20-19:33:13.508084 140008266061376 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062176.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-19:33:14.656782 140008266061376 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062176.sst): second pass time = 1.33's, 81.651'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.15's, 34.885'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2706.699'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.291'MB/sec write SST data time = 0.36's, 111.976'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 35282.745'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.651'MB/sec zip pipeline throughput = 163.214'MB/sec entries = 923028 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.41 avg-zval = 37.18 usrkeys = 923028 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.41 avg-zval = 37.18 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923028 bytes = 2999856 } seq expand size = 6461196 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0343 GB dict = 1.08 MB all = 0.0402 GB } UnZip/Zip{ index = 10.5142 value = 3.1576 dict = 1.00 all = 3.3416 } Zip/UnZip{ index = 0.0951 value = 0.3167 dict = 1.00 all = 0.2993 } ---------------------------- total value len = 0.336916 GB avg = 0.121 KB (by entry num) total key len = 0.077680 GB avg = 0.028 KB total ukey len = 0.055486 GB avg = 0.020 KB total ukey num = 0.002774302 Billion total entry num = 0.002774302 Billion write speed all = 0.010704020 MB/sec (with seq num) write speed all = 0.010131007 MB/sec (without seq num) 2024/11/20-19:33:14.731329 140008266061376 [compaction_job.cc:1948] [default] [JOB 8284] Generated table #62176: 923028 keys, 40165986 bytes, temperature: kWarm 2024/11/20-19:33:14.731421 140008266061376 EVENT_LOG_v1 {"time_micros": 1732102394731376, "cf_name": "default", "job": 8284, "event": "table_file_creation", "file_number": 62176, "file_size": 40165986, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 979921761, "largest_seqno": 1024549082, "table_properties": {"data_size": 34321472, "index_size": 1755776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25844784, "raw_average_key_size": 28, "raw_value_size": 108372964, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923028, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732102391, "oldest_key_time": 0, "file_creation_time": 1732102391, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 62176, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:33:14.775746 140008266061376 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062177.sst) /dev/shm/ToplingTemp/Topling-891171-Fg2t0l 2024/11/20-19:33:15.769754 140008266061376 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062177.sst): first pass time = 0.99's, 135.032'MB/sec 2024/11/20-19:33:17.001569 140008266061376 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062177.sst): old prealloc_size = 147639500, real_size = 36124512 2024/11/20-19:33:17.002152 140008266061376 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062177.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923056 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0341 GB avg-val = 117.41 avg-zval = 36.98 2024/11/20-19:33:17.002927 140008266061376 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062177.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-19:33:18.136134 140008266061376 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062177.sst): second pass time = 1.23's, 88.038'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.13's, 35.240'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3103.896'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.264'MB/sec write SST data time = 0.26's, 153.028'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 31049.966'MB/sec zip my value time = 1.23's, unzip length = 0.108'GB zip my value throughput = 88.038'MB/sec zip pipeline throughput = 96.174'MB/sec entries = 923056 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.41 avg-zval = 36.98 usrkeys = 923056 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.41 avg-zval = 36.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923056 bytes = 2999952 } seq expand size = 6461392 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0341 GB dict = 1.08 MB all = 0.0400 GB } UnZip/Zip{ index = 10.5130 value = 3.1746 dict = 1.00 all = 3.3569 } Zip/UnZip{ index = 0.0951 value = 0.3150 dict = 1.00 all = 0.2979 } ---------------------------- total value len = 0.451750 GB avg = 0.122 KB (by entry num) total key len = 0.103526 GB avg = 0.028 KB total ukey len = 0.073947 GB avg = 0.020 KB total ukey num = 0.003697358 Billion total entry num = 0.003697358 Billion write speed all = 0.014334773 MB/sec (with seq num) write speed all = 0.013571178 MB/sec (without seq num) 2024/11/20-19:33:18.187202 140008266061376 [compaction_job.cc:1948] [default] [JOB 8284] Generated table #62177: 923056 keys, 39982062 bytes, temperature: kWarm 2024/11/20-19:33:18.187273 140008266061376 EVENT_LOG_v1 {"time_micros": 1732102398187241, "cf_name": "default", "job": 8284, "event": "table_file_creation", "file_number": 62177, "file_size": 39982062, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 979921814, "largest_seqno": 1024548572, "table_properties": {"data_size": 34137552, "index_size": 1756032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25845568, "raw_average_key_size": 28, "raw_value_size": 108372212, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923056, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732102394, "oldest_key_time": 0, "file_creation_time": 1732102394, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 62177, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:33:18.225641 140008266061376 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062178.sst) /dev/shm/ToplingTemp/Topling-891171-Zk9hbp 2024/11/20-19:33:19.215839 140008266061376 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062178.sst): first pass time = 0.99's, 135.551'MB/sec 2024/11/20-19:33:20.564741 140008266061376 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062178.sst): old prealloc_size = 147639500, real_size = 36592048 2024/11/20-19:33:20.565376 140008266061376 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062178.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925662 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.95 raw-val = 0.1083 GB zip-val = 0.0346 GB avg-val = 117.00 avg-zval = 37.33 2024/11/20-19:33:20.566162 140008266061376 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062178.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-19:33:21.632603 140008266061376 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062178.sst): second pass time = 1.35's, 80.335'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.07's, 37.997'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2956.455'MB/sec (index lex order gen) rebuild zvType time = 0.84's, 0.276'MB/sec write SST data time = 0.23's, 178.163'MB/sec dict compress time = 0.00's, 1080814.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 17691.580'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.335'MB/sec zip pipeline throughput = 80.335'MB/sec entries = 925662 avg-key = 28.00 avg-zkey = 1.95 avg-val = 117.00 avg-zval = 37.33 usrkeys = 925662 avg-key = 20.00 avg-zkey = 1.95 avg-val = 124.00 avg-zval = 37.33 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925662 bytes = 3124128 } seq expand size = 6479634 multi value expand size = 0 cnt WriteAppend = 46 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0346 GB dict = 1.08 MB all = 0.0406 GB } UnZip/Zip{ index = 10.2414 value = 3.1343 dict = 1.00 all = 3.3079 } Zip/UnZip{ index = 0.0976 value = 0.3190 dict = 1.00 all = 0.3023 } ---------------------------- total value len = 0.566529 GB avg = 0.123 KB (by entry num) total key len = 0.129445 GB avg = 0.028 KB total ukey len = 0.092460 GB avg = 0.020 KB total ukey num = 0.004623020 Billion total entry num = 0.004623020 Billion write speed all = 0.017965337 MB/sec (with seq num) write speed all = 0.017010655 MB/sec (without seq num) 2024/11/20-19:33:21.682228 140008266061376 [compaction_job.cc:1948] [default] [JOB 8284] Generated table #62178: 925662 keys, 40575038 bytes, temperature: kWarm 2024/11/20-19:33:21.682288 140008266061376 EVENT_LOG_v1 {"time_micros": 1732102401682259, "cf_name": "default", "job": 8284, "event": "table_file_creation", "file_number": 62178, "file_size": 40575038, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 919911853, "largest_seqno": 1024548983, "table_properties": {"data_size": 34552800, "index_size": 1807680, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25918536, "raw_average_key_size": 28, "raw_value_size": 108299228, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925662, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732102398, "oldest_key_time": 0, "file_creation_time": 1732102398, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 62178, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:33:21.720742 140008266061376 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062179.sst) /dev/shm/ToplingTemp/Topling-891171-1evZIn 2024/11/20-19:33:22.556854 140008266061376 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062179.sst): first pass time = 0.84's, 160.533'MB/sec 2024/11/20-19:33:23.983689 140008266061376 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062179.sst): old prealloc_size = 147639500, real_size = 36504048 2024/11/20-19:33:23.984353 140008266061376 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062179.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926929 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.80 avg-zval = 37.19 2024/11/20-19:33:23.984654 140008266061376 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062179.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-19:33:25.001573 140008266061376 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062179.sst): second pass time = 1.43's, 75.919'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.02's, 37.587'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2819.005'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.309'MB/sec write SST data time = 0.27's, 143.637'MB/sec dict compress time = 0.00's, 1080116.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 20022.170'MB/sec zip my value time = 1.43's, unzip length = 0.108'GB zip my value throughput = 75.919'MB/sec zip pipeline throughput = 75.919'MB/sec entries = 926929 avg-key = 28.00 avg-zkey = 1.94 avg-val = 116.80 avg-zval = 37.19 usrkeys = 926929 avg-key = 20.00 avg-zkey = 1.94 avg-val = 118.55 avg-zval = 37.19 TagRS{ kinds = RS_Key0_TagN bytes = 144920 } TagArray{ size = 231311 bytes = 751776 } seq expand size = 1619177 multi value expand size = 0 cnt WriteAppend = 37 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.3076 value = 3.1405 dict = 1.00 all = 3.5082 } Zip/UnZip{ index = 0.0970 value = 0.3184 dict = 1.00 all = 0.2850 } ---------------------------- total value len = 0.676412 GB avg = 0.122 KB (by entry num) total key len = 0.155399 GB avg = 0.028 KB total ukey len = 0.110999 GB avg = 0.020 KB total ukey num = 0.005549949 Billion total entry num = 0.005549949 Billion write speed all = 0.021469865 MB/sec (with seq num) write speed all = 0.020323867 MB/sec (without seq num) 2024/11/20-19:33:25.048991 140008266061376 [compaction_job.cc:1948] [default] [JOB 8284] Generated table #62179: 926929 keys, 38258755 bytes, temperature: kWarm 2024/11/20-19:33:25.049069 140008266061376 EVENT_LOG_v1 {"time_micros": 1732102405049027, "cf_name": "default", "job": 8284, "event": "table_file_creation", "file_number": 62179, "file_size": 38258755, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 973933750, "table_properties": {"data_size": 34473632, "index_size": 1798528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25954012, "raw_average_key_size": 28, "raw_value_size": 108263842, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926929, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732102401, "oldest_key_time": 0, "file_creation_time": 1732102401, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 62179, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:33:25.086563 140008266061376 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062180.sst) /dev/shm/ToplingTemp/Topling-891171-qIKi6l 2024/11/20-19:33:25.959096 140008266061376 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062180.sst): first pass time = 0.87's, 153.831'MB/sec 2024/11/20-19:33:27.394345 140008266061376 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062180.sst): old prealloc_size = 147639500, real_size = 36331296 2024/11/20-19:33:27.394949 140008266061376 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062180.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926749 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0343 GB avg-val = 116.83 avg-zval = 37.05 2024/11/20-19:33:27.395275 140008266061376 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062180.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-19:33:28.102446 140008266061376 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062180.sst): second pass time = 1.43's, 75.480'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.71's, 53.793'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3046.582'MB/sec (index lex order gen) rebuild zvType time = 0.58's, 0.403'MB/sec write SST data time = 0.13's, 288.262'MB/sec dict compress time = 0.00's, 1080198.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 18791.281'MB/sec zip my value time = 1.43's, unzip length = 0.108'GB zip my value throughput = 75.480'MB/sec zip pipeline throughput = 75.480'MB/sec entries = 926749 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.83 avg-zval = 37.05 usrkeys = 926749 avg-key = 20.00 avg-zkey = 1.90 avg-val = 118.58 avg-zval = 37.05 TagRS{ kinds = RS_Key0_TagN bytes = 144920 } TagArray{ size = 232578 bytes = 755888 } seq expand size = 1628046 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0343 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.5110 value = 3.1532 dict = 1.00 all = 3.5237 } Zip/UnZip{ index = 0.0951 value = 0.3171 dict = 1.00 all = 0.2838 } ---------------------------- total value len = 0.786309 GB avg = 0.121 KB (by entry num) total key len = 0.181348 GB avg = 0.028 KB total ukey len = 0.129534 GB avg = 0.020 KB total ukey num = 0.006476698 Billion total entry num = 0.006476698 Billion write speed all = 0.024974183 MB/sec (with seq num) write speed all = 0.023636930 MB/sec (without seq num) 2024/11/20-19:33:28.151488 140008266061376 [compaction_job.cc:1948] [default] [JOB 8284] Generated table #62180: 926749 keys, 38090133 bytes, temperature: kWarm 2024/11/20-19:33:28.151558 140008266061376 EVENT_LOG_v1 {"time_micros": 1732102408151526, "cf_name": "default", "job": 8284, "event": "table_file_creation", "file_number": 62180, "file_size": 38090133, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 973933812, "table_properties": {"data_size": 34336016, "index_size": 1763392, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25948972, "raw_average_key_size": 28, "raw_value_size": 108268810, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926749, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732102405, "oldest_key_time": 0, "file_creation_time": 1732102405, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 62180, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:33:28.190330 140008266061376 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062181.sst) /dev/shm/ToplingTemp/Topling-891171-aD3Xam 2024/11/20-19:33:29.002104 140008266061376 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062181.sst): first pass time = 0.81's, 165.349'MB/sec 2024/11/20-19:33:30.324292 140008266061376 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062181.sst): old prealloc_size = 147639500, real_size = 36473040 2024/11/20-19:33:30.324981 140008266061376 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062181.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926934 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.97 raw-val = 0.1083 GB zip-val = 0.0344 GB avg-val = 116.80 avg-zval = 37.13 2024/11/20-19:33:30.325158 140008266061376 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062181.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-19:33:31.031189 140008266061376 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062181.sst): second pass time = 1.32's, 81.935'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.71's, 53.126'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2828.660'MB/sec (index lex order gen) rebuild zvType time = 0.57's, 0.404'MB/sec write SST data time = 0.13's, 281.806'MB/sec dict compress time = 0.00's, 1080080.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 18556.163'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.935'MB/sec zip pipeline throughput = 81.935'MB/sec entries = 926934 avg-key = 28.00 avg-zkey = 1.97 avg-val = 116.80 avg-zval = 37.13 usrkeys = 926934 avg-key = 20.00 avg-zkey = 1.97 avg-val = 116.99 avg-zval = 37.13 TagRS{ kinds = RS_Key0_TagN bytes = 144920 } TagArray{ size = 25255 bytes = 78944 } seq expand size = 176785 multi value expand size = 0 cnt WriteAppend = 25 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.1777 value = 3.1454 dict = 1.00 all = 3.5739 } Zip/UnZip{ index = 0.0983 value = 0.3179 dict = 1.00 all = 0.2798 } ---------------------------- total value len = 0.894749 GB avg = 0.121 KB (by entry num) total key len = 0.207302 GB avg = 0.028 KB total ukey len = 0.148073 GB avg = 0.020 KB total ukey num = 0.007403632 Billion total entry num = 0.007403632 Billion write speed all = 0.028440616 MB/sec (with seq num) write speed all = 0.026912092 MB/sec (without seq num) 2024/11/20-19:33:31.085105 140008266061376 [compaction_job.cc:1948] [default] [JOB 8284] Generated table #62181: 926934 keys, 37554847 bytes, temperature: kWarm 2024/11/20-19:33:31.085180 140008266061376 EVENT_LOG_v1 {"time_micros": 1732102411085146, "cf_name": "default", "job": 8284, "event": "table_file_creation", "file_number": 62181, "file_size": 37554847, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 965663958, "table_properties": {"data_size": 34419648, "index_size": 1821504, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25954152, "raw_average_key_size": 28, "raw_value_size": 108263652, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926934, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732102408, "oldest_key_time": 0, "file_creation_time": 1732102408, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 62181, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:33:31.124324 140008266061376 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062182.sst) /dev/shm/ToplingTemp/Topling-891171-5XZTzo 2024/11/20-19:33:31.904375 140008266061376 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062182.sst): first pass time = 0.78's, 172.074'MB/sec 2024/11/20-19:33:33.262991 140008266061376 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062182.sst): old prealloc_size = 147639500, real_size = 36414352 2024/11/20-19:33:33.263654 140008266061376 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062182.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926563 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1083 GB zip-val = 0.0344 GB avg-val = 116.86 avg-zval = 37.11 2024/11/20-19:33:33.263826 140008266061376 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062182.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-19:33:33.965164 140008266061376 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062182.sst): second pass time = 1.36's, 79.750'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.70's, 53.478'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3001.012'MB/sec (index lex order gen) rebuild zvType time = 0.57's, 0.406'MB/sec write SST data time = 0.13's, 285.851'MB/sec dict compress time = 0.00's, 1080640.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 18250.046'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.750'MB/sec zip pipeline throughput = 79.750'MB/sec entries = 926563 avg-key = 28.00 avg-zkey = 1.94 avg-val = 116.86 avg-zval = 37.11 usrkeys = 926563 avg-key = 20.00 avg-zkey = 1.94 avg-val = 117.19 avg-zval = 37.11 TagRS{ kinds = RS_Key0_TagN bytes = 144880 } TagArray{ size = 44351 bytes = 133072 } seq expand size = 310457 multi value expand size = 0 cnt WriteAppend = 19 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.3083 value = 3.1489 dict = 1.00 all = 3.5743 } Zip/UnZip{ index = 0.0970 value = 0.3176 dict = 1.00 all = 0.2798 } ---------------------------- total value len = 1.003334 GB avg = 0.120 KB (by entry num) total key len = 0.233245 GB avg = 0.028 KB total ukey len = 0.166604 GB avg = 0.020 KB total ukey num = 0.008330195 Billion total entry num = 0.008330195 Billion write speed all = 0.031909970 MB/sec (with seq num) write speed all = 0.030190282 MB/sec (without seq num) 2024/11/20-19:33:34.010610 140008266061376 [compaction_job.cc:1948] [default] [JOB 8284] Generated table #62182: 926563 keys, 37550799 bytes, temperature: kWarm 2024/11/20-19:33:34.010679 140008266061376 EVENT_LOG_v1 {"time_micros": 1732102414010645, "cf_name": "default", "job": 8284, "event": "table_file_creation", "file_number": 62182, "file_size": 37550799, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 960329456, "table_properties": {"data_size": 34384832, "index_size": 1797696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25943764, "raw_average_key_size": 28, "raw_value_size": 108274086, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926563, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732102411, "oldest_key_time": 0, "file_creation_time": 1732102411, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 62182, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:33:34.055800 140008266061376 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062183.sst) /dev/shm/ToplingTemp/Topling-891171-hbweio 2024/11/20-19:33:35.007837 140008266061376 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062183.sst): first pass time = 0.95's, 140.987'MB/sec 2024/11/20-19:33:36.184196 140008266061376 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062183.sst): old prealloc_size = 147639500, real_size = 36577200 2024/11/20-19:33:36.184810 140008266061376 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062183.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923229 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1084 GB zip-val = 0.0346 GB avg-val = 117.38 avg-zval = 37.48 2024/11/20-19:33:36.185599 140008266061376 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062183.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-19:33:36.983314 140008266061376 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062183.sst): second pass time = 1.18's, 92.194'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.80's, 50.644'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3041.860'MB/sec (index lex order gen) rebuild zvType time = 0.66's, 0.350'MB/sec write SST data time = 0.14's, 290.490'MB/sec dict compress time = 0.00's, 1082786.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33372.970'MB/sec zip my value time = 1.18's, unzip length = 0.108'GB zip my value throughput = 92.194'MB/sec zip pipeline throughput = 92.194'MB/sec entries = 923229 avg-key = 28.00 avg-zkey = 1.89 avg-val = 117.38 avg-zval = 37.48 usrkeys = 923229 avg-key = 20.00 avg-zkey = 1.89 avg-val = 123.87 avg-zval = 37.48 TagRS{ kinds = RS_Key0_TagN bytes = 144360 } TagArray{ size = 855621 bytes = 2887728 } seq expand size = 5989347 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0346 GB dict = 1.08 MB all = 0.0405 GB } UnZip/Zip{ index = 10.5824 value = 3.1319 dict = 1.00 all = 3.3164 } Zip/UnZip{ index = 0.0945 value = 0.3193 dict = 1.00 all = 0.3015 } ---------------------------- total value len = 1.117690 GB avg = 0.121 KB (by entry num) total key len = 0.259096 GB avg = 0.028 KB total ukey len = 0.185068 GB avg = 0.020 KB total ukey num = 0.009253424 Billion total entry num = 0.009253424 Billion write speed all = 0.035525253 MB/sec (with seq num) write speed all = 0.033615122 MB/sec (without seq num) 2024/11/20-19:33:37.031958 140008266061376 [compaction_job.cc:1948] [default] [JOB 8284] Generated table #62183: 923229 keys, 40470834 bytes, temperature: kWarm 2024/11/20-19:33:37.032029 140008266061376 EVENT_LOG_v1 {"time_micros": 1732102417031996, "cf_name": "default", "job": 8284, "event": "table_file_creation", "file_number": 62183, "file_size": 40470834, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1024548618, "table_properties": {"data_size": 34601376, "index_size": 1744832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25850412, "raw_average_key_size": 28, "raw_value_size": 108367362, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923229, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732102414, "oldest_key_time": 0, "file_creation_time": 1732102414, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 62183, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:33:37.069794 140008266061376 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062184.sst) /dev/shm/ToplingTemp/Topling-891171-FieuZl 2024/11/20-19:33:37.347024 140008266061376 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062184.sst): first pass time = 0.28's, 249.173'MB/sec 2024/11/20-19:33:37.954037 140008266061376 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062184.sst): old prealloc_size = 147639500, real_size = 19359056 2024/11/20-19:33:37.954465 140008266061376 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062184.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 474907 prefix = 4 raw-key = 0.0076 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0558 GB zip-val = 0.0183 GB avg-val = 117.43 avg-zval = 38.61 2024/11/20-19:33:37.954885 140008266061376 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062184.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.069 GB 2024/11/20-19:33:38.379822 140008266061376 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00/062184.sst): second pass time = 0.61's, 92.016'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.43's, 50.134'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2638.554'MB/sec (index lex order gen) rebuild zvType time = 0.36's, 0.334'MB/sec write SST data time = 0.07's, 305.833'MB/sec dict compress time = 0.00's, 558730.000'MB/sec z-dict build time = 0.00's, sample length = 0.559'MB, throughput = 16852.059'MB/sec zip my value time = 0.61's, unzip length = 0.056'GB zip my value throughput = 92.016'MB/sec zip pipeline throughput = 92.016'MB/sec entries = 474907 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.43 avg-zval = 38.61 usrkeys = 474907 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.43 avg-zval = 38.61 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 474907 bytes = 1543456 } seq expand size = 3324349 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0095 GB value = 0.0558 GB dict = 0.56 MB all = 0.0691 GB } __ZipSize{ index = 0.0009 GB value = 0.0183 GB dict = 0.56 MB all = 0.0213 GB } UnZip/Zip{ index = 10.5008 value = 3.0416 dict = 1.00 all = 3.2356 } Zip/UnZip{ index = 0.0952 value = 0.3288 dict = 1.00 all = 0.3091 } ---------------------------- total value len = 1.176784 GB avg = 0.121 KB (by entry num) total key len = 0.272393 GB avg = 0.028 KB total ukey len = 0.194567 GB avg = 0.020 KB total ukey num = 0.009728331 Billion total entry num = 0.009728331 Billion write speed all = 0.037391807 MB/sec (with seq num) write speed all = 0.035383717 MB/sec (without seq num) 2024/11/20-19:33:38.411657 140008266061376 [compaction_job.cc:1948] [default] [JOB 8284] Generated table #62184: 474907 keys, 21346073 bytes, temperature: kWarm 2024/11/20-19:33:38.411723 140008266061376 EVENT_LOG_v1 {"time_micros": 1732102418411692, "cf_name": "default", "job": 8284, "event": "table_file_creation", "file_number": 62184, "file_size": 21346073, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 980002266, "largest_seqno": 1024547666, "table_properties": {"data_size": 18335616, "index_size": 904512, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 13297396, "raw_average_key_size": 28, "raw_value_size": 55769058, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 474907, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732102417, "oldest_key_time": 0, "file_creation_time": 1732102417, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 62184, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:33:38.429633 140008266061376 [compaction_job.cc:703] job-08284: subcompact[0], size: 0.411769 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08284/att-00 [062174,062175,062176,062177,062178,062179,062180,062181,062182,062183,062184] 2024/11/20-19:33:38.502888 140008266061376 (Original Log Time 2024/11/20-19:33:03.909243) EVENT_LOG_v1 {"time_micros": 1732102383909199, "job": 8284, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [62141], "files_L3": [62110, 62111, 62112, 62113, 62114, 62056, 62057, 62058, 62059, 62060, 62161], "score": 0.813857, "input_data_size": 435069324, "oldest_snapshot_seqno": 1024773844} 2024/11/20-19:33:38.502891 140008266061376 (Original Log Time 2024/11/20-19:33:38.501829) [compaction_job.cc:2022] [default] [JOB 8284] Compacted 1@2 + 11@3 files to L3 => 411768573 bytes 2024/11/20-19:33:38.502892 140008266061376 (Original Log Time 2024/11/20-19:33:38.502817) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 11 0 0 0] max score 0.02, MB/sec: 12.6 rd, 11.9 wr, level 3, files in(1, 11) out(11 +0 blob) MB in(27.2, 387.7 +0.0 blob) out(392.7 +0.0 blob), read-write-amplify(29.7) write-amplify(14.4) OK, records in: 9838152, records dropped: 109821 output_compression: Snappy 2024/11/20-19:33:38.502896 140008266061376 (Original Log Time 2024/11/20-19:33:38.502864) EVENT_LOG_v1 {"time_micros": 1732102418502840, "cf": "default", "job": 8284, "event": "compaction_finished", "compaction_time_micros": 34520398, "compaction_time_cpu_micros": 15122954, "output_level": 3, "num_output_files": 11, "total_output_size": 411768573, "num_input_records": 9838152, "num_output_records": 9728331, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 333559490, "file_range_sync_nanos": 0, "file_fsync_nanos": 9715, "file_prepare_write_nanos": 37851, "lsm_state": [0, 0, 0, 11, 0, 0, 0]} 2024/11/20-19:33:38.626566 140008266061376 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732096383,"starts":1732102383,"executesMs":34773,"compactionJobId":8284,"attempt":0,"compactionInputRawBytes":1426839012,"compactionInputZipBytes":435069324,"compactionOutputRawBytes":0,"compactionOutputZipBytes":411768573,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-70439395eceb8a9b68842e144a83e75d-d02218f16d3998d1-00"}}, response = 2024/11/20-19:33:38.626655 140008266061376 INFO dcompact_worker.cpp:1464: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-08284/att-00: olev 3, work 34.774 s, result 7.678 ms, install 1.082 ms, input{raw 1.329 GiB zip 414.914 MiB}