2024/08/20-01:08:46.875245 140500296816192 DEBG dcompact_worker.cpp:2211: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00: fork to child time = 0.023364 sec 2024/08/20-01:08:46.876367 140500296816192 DEBG dcompact_worker.cpp:1011: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00 2024/08/20-01:08:46.881502 140500296816192 DEBG dcompact_worker.cpp:1031: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00 2024/08/20-01:08:46.883317 140500296816192 TRAC dcompact_worker.cpp:1081: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/08/20-01:08:46.883615 140500296816192 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/08/20-01:08:46.925686 140500296816192 [version_set.cc:6139] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-08-19T23.41.30/.rocksdb/job-04063/att-00/MANIFEST-14619535 2024/08/20-01:08:46.946208 140500296816192 [version_set.cc:6182] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-08-19T23.41.30/.rocksdb/job-04063/att-00/MANIFEST-14619535 succeeded,manifest_file_number is 14619535, next_file_number is 14624242, last_sequence is 189684786762, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 14624204 2024/08/20-01:08:46.946214 140500296816192 [version_set.cc:6197] Column family [default] (ID 0), log number is 0 2024/08/20-01:08:46.946278 140500296816192 [version_set.cc:5652] Creating manifest 14624242 2024/08/20-01:08:46.979994 140500296816192 DEBG dcompact_worker.cpp:1215: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00: bottommost_level: fake = 1, rpc = 0 2024/08/20-01:08:46.994418 140500296816192 [compaction_job.cc:2433] [default] [JOB 4063] Compacting 1@1 + 6@2 files to L2, score 0.82, subcompactions 1 : 1 2024/08/20-01:08:46.994433 140500296816192 [compaction_job.cc:2441] [default]: Compaction start summary: Base version 2013 Base level 1, inputs: [14624234(31MB)], [14624193(20MB) 14624194(20MB) 14624195(20MB) 14624196(20MB) 14624197(20MB) 14624198(20MB)] 2024/08/20-01:08:47.227538 140500296816192 [top_zip_table.cc:59] core git_version_hash_info_is:commit 7517bad5c1a147474c3830831b9a22c996321ae8 Author: leipeng Date: 2024-01-08 19:18:17 +0800 vfork_cmd: call close before wait_finish g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/08/20-01:08:47.227570 140500296816192 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit 7517bad5c1a147474c3830831b9a22c996321ae8 Author: leipeng Date: 2024-01-08 19:18:17 +0800 vfork_cmd: call close before wait_finish g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/08/20-01:08:47.227593 140500296816192 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit 7517bad5c1a147474c3830831b9a22c996321ae8 Author: leipeng Date: 2024-01-08 19:18:17 +0800 vfork_cmd: call close before wait_finish g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/08/20-01:08:47.227605 140500296816192 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit 7f80399bfbd2071b6e0f5683e8ab7225b8e6e394 Author: leipeng Date: 2024-01-19 14:52:30 +0800 Change ToplingZipTableOptions::sampleRatio to vector g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/08/20-01:08:47.227615 140500296816192 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit 1240987ca13d0fe562c9b9c037155079f0877840 Author: leipeng Date: 2024-01-19 15:07:04 +0800 Change ToplingZipTableOptions::sampleRatio to vector g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/08/20-01:08:47.402275 140500296816192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624243.sst) /dev/shm/ToplingTemp/Topling-88878-cG5FvH 2024/08/20-01:08:47.712610 140500296816192 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624243.sst): first pass time = 0.31's, 216.282'MB/sec 2024/08/20-01:08:50.352343 140500296816192 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624243.sst): old prealloc_size = 73819750, real_size = 19318240 2024/08/20-01:08:50.352701 140500296816192 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624243.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 463065 prefix = 4 raw-key = 0.0074 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0541 GB zip-val = 0.0183 GB avg-val = 116.92 avg-zval = 39.55 2024/08/20-01:08:50.353180 140500296816192 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624243.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.020 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.067 GB 2024/08/20-01:08:50.992171 140500296816192 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624243.sst): second pass time = 2.64's, 20.519'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 33.224'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2655.011'MB/sec (index lex order gen) rebuild zvType time = 0.55's, 0.209'MB/sec write SST data time = 0.09's, 247.009'MB/sec dict compress time = 0.00's, 545338.000'MB/sec z-dict build time = 0.00's, sample length = 0.545'MB, throughput = 15632.001'MB/sec zip my value time = 2.64's, unzip length = 0.054'GB zip my value throughput = 20.519'MB/sec zip pipeline throughput = 101.088'MB/sec entries = 463065 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.92 avg-zval = 39.55 usrkeys = 463065 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.92 avg-zval = 39.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 463065 bytes = 1504976 } seq expand size = 3241455 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0093 GB value = 0.0541 GB dict = 0.55 MB all = 0.0671 GB } __ZipSize{ index = 0.0009 GB value = 0.0183 GB dict = 0.55 MB all = 0.0213 GB } UnZip/Zip{ index = 10.4362 value = 2.9562 dict = 1.00 all = 3.1568 } Zip/UnZip{ index = 0.0958 value = 0.3383 dict = 1.00 all = 0.3168 } ---------------------------- total value len = 62.354134 GB avg = 0.180 KB (by entry num) total key len = 6.801723 GB avg = 0.020 KB total ukey len = 3.198769 GB avg = 0.008 KB total ukey num = 0.389796626 Billion total entry num = 0.346674634 Billion write speed all = 2.534129379 MB/sec (with seq num) write speed all = 2.432501733 MB/sec (without seq num) 2024/08/20-01:08:51.022443 140500296816192 [compaction_job.cc:1944] [default] [JOB 4063] Generated table #14624243: 463065 keys, 21258250 bytes, temperature: kWarm 2024/08/20-01:08:51.022546 140500296816192 EVENT_LOG_v1 {"time_micros": 1724087331022485, "cf_name": "default", "job": 4063, "event": "table_file_creation", "file_number": 14624243, "file_size": 21258250, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189638867654, "largest_seqno": 189684748908, "table_properties": {"data_size": 18314896, "index_size": 887424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 12965820, "raw_average_key_size": 28, "raw_value_size": 54143110, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 463065, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1724087327, "oldest_key_time": 0, "file_creation_time": 1724087327, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14624243, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-01:08:51.119206 140500296816192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624244.sst) /dev/shm/ToplingTemp/Topling-88878-9s0gSI 2024/08/20-01:08:51.928060 140500296816192 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624244.sst): first pass time = 0.81's, 82.973'MB/sec 2024/08/20-01:08:56.817088 140500296816192 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624244.sst): old prealloc_size = 73819750, real_size = 19078048 2024/08/20-01:08:56.817851 140500296816192 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624244.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 462975 prefix = 4 raw-key = 0.0074 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0541 GB zip-val = 0.0181 GB avg-val = 116.95 avg-zval = 39.05 2024/08/20-01:08:56.818735 140500296816192 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624244.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.020 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.067 GB 2024/08/20-01:08:57.490027 140500296816192 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624244.sst): second pass time = 4.89's, 11.078'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.67's, 31.231'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1219.174'MB/sec (index lex order gen) rebuild zvType time = 0.54's, 0.214'MB/sec write SST data time = 0.13's, 159.768'MB/sec dict compress time = 0.00's, 545600.000'MB/sec z-dict build time = 0.00's, sample length = 0.546'MB, throughput = 4891.957'MB/sec zip my value time = 4.89's, unzip length = 0.054'GB zip my value throughput = 11.078'MB/sec zip pipeline throughput = 150.886'MB/sec entries = 462975 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.95 avg-zval = 39.05 usrkeys = 462975 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.95 avg-zval = 39.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 462975 bytes = 1504688 } seq expand size = 3240825 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0093 GB value = 0.0541 GB dict = 0.55 MB all = 0.0671 GB } __ZipSize{ index = 0.0009 GB value = 0.0181 GB dict = 0.55 MB all = 0.0210 GB } UnZip/Zip{ index = 10.5054 value = 2.9947 dict = 1.00 all = 3.1932 } Zip/UnZip{ index = 0.0952 value = 0.3339 dict = 1.00 all = 0.3132 } ---------------------------- total value len = 62.411521 GB avg = 0.180 KB (by entry num) total key len = 6.814687 GB avg = 0.020 KB total ukey len = 3.208028 GB avg = 0.008 KB total ukey num = 0.390259601 Billion total entry num = 0.347137609 Billion write speed all = 2.536103398 MB/sec (with seq num) write speed all = 2.434364256 MB/sec (without seq num) 2024/08/20-01:08:57.521663 140500296816192 [compaction_job.cc:1944] [default] [JOB 4063] Generated table #14624244: 462975 keys, 21016080 bytes, temperature: kWarm 2024/08/20-01:08:57.521747 140500296816192 EVENT_LOG_v1 {"time_micros": 1724087337521702, "cf_name": "default", "job": 4063, "event": "table_file_creation", "file_number": 14624244, "file_size": 21016080, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189636721631, "largest_seqno": 189684752951, "table_properties": {"data_size": 18080720, "index_size": 881408, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 12963300, "raw_average_key_size": 28, "raw_value_size": 54145642, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 462975, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1724087331, "oldest_key_time": 0, "file_creation_time": 1724087331, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14624244, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-01:08:57.558309 140500296816192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624245.sst) /dev/shm/ToplingTemp/Topling-88878-KAr0hJ 2024/08/20-01:08:58.486590 140500296816192 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624245.sst): first pass time = 0.93's, 72.298'MB/sec 2024/08/20-01:09:00.890806 140500296816192 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624245.sst): old prealloc_size = 73819750, real_size = 19119968 2024/08/20-01:09:00.891191 140500296816192 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624245.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 463249 prefix = 4 raw-key = 0.0074 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0541 GB zip-val = 0.0181 GB avg-val = 116.87 avg-zval = 39.12 2024/08/20-01:09:00.891636 140500296816192 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624245.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.020 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.067 GB 2024/08/20-01:09:01.582750 140500296816192 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624245.sst): second pass time = 2.40's, 22.542'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.69's, 30.434'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2456.842'MB/sec (index lex order gen) rebuild zvType time = 0.54's, 0.213'MB/sec write SST data time = 0.15's, 141.999'MB/sec dict compress time = 0.00's, 544998.000'MB/sec z-dict build time = 0.00's, sample length = 0.545'MB, throughput = 15225.109'MB/sec zip my value time = 2.40's, unzip length = 0.054'GB zip my value throughput = 22.542'MB/sec zip pipeline throughput = 150.536'MB/sec entries = 463249 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.87 avg-zval = 39.12 usrkeys = 463249 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.87 avg-zval = 39.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 463249 bytes = 1505568 } seq expand size = 3242743 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0093 GB value = 0.0541 GB dict = 0.54 MB all = 0.0671 GB } __ZipSize{ index = 0.0009 GB value = 0.0181 GB dict = 0.54 MB all = 0.0211 GB } UnZip/Zip{ index = 10.5032 value = 2.9874 dict = 1.00 all = 3.1868 } Zip/UnZip{ index = 0.0952 value = 0.3347 dict = 1.00 all = 0.3138 } ---------------------------- total value len = 62.468902 GB avg = 0.180 KB (by entry num) total key len = 6.827658 GB avg = 0.020 KB total ukey len = 3.217293 GB avg = 0.008 KB total ukey num = 0.390722850 Billion total entry num = 0.347600858 Billion write speed all = 2.538300150 MB/sec (with seq num) write speed all = 2.436440512 MB/sec (without seq num) 2024/08/20-01:09:01.637517 140500296816192 [compaction_job.cc:1944] [default] [JOB 4063] Generated table #14624245: 463249 keys, 21058230 bytes, temperature: kWarm 2024/08/20-01:09:01.637603 140500296816192 EVENT_LOG_v1 {"time_micros": 1724087341637558, "cf_name": "default", "job": 4063, "event": "table_file_creation", "file_number": 14624245, "file_size": 21058230, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189636721676, "largest_seqno": 189684753511, "table_properties": {"data_size": 18121872, "index_size": 882112, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 12970972, "raw_average_key_size": 28, "raw_value_size": 54138022, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 463249, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1724087337, "oldest_key_time": 0, "file_creation_time": 1724087337, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14624245, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-01:09:01.672038 140500296816192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624246.sst) /dev/shm/ToplingTemp/Topling-88878-P8IJsI 2024/08/20-01:09:02.315451 140500296816192 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624246.sst): first pass time = 0.64's, 104.312'MB/sec 2024/08/20-01:09:03.979680 140500296816192 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624246.sst): old prealloc_size = 73819750, real_size = 19126592 2024/08/20-01:09:03.980071 140500296816192 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624246.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 463429 prefix = 4 raw-key = 0.0074 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0541 GB zip-val = 0.0181 GB avg-val = 116.81 avg-zval = 39.12 2024/08/20-01:09:03.980491 140500296816192 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624246.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.020 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.067 GB 2024/08/20-01:09:04.413404 140500296816192 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624246.sst): second pass time = 1.66's, 32.648'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.43's, 48.571'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2411.057'MB/sec (index lex order gen) rebuild zvType time = 0.35's, 0.327'MB/sec write SST data time = 0.08's, 264.740'MB/sec dict compress time = 0.00's, 544956.000'MB/sec z-dict build time = 0.00's, sample length = 0.545'MB, throughput = 11986.275'MB/sec zip my value time = 1.66's, unzip length = 0.054'GB zip my value throughput = 32.648'MB/sec zip pipeline throughput = 89.056'MB/sec entries = 463429 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.81 avg-zval = 39.12 usrkeys = 463429 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.81 avg-zval = 39.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 463429 bytes = 1506160 } seq expand size = 3244003 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0093 GB value = 0.0541 GB dict = 0.54 MB all = 0.0671 GB } __ZipSize{ index = 0.0009 GB value = 0.0181 GB dict = 0.54 MB all = 0.0211 GB } UnZip/Zip{ index = 10.5004 value = 2.9862 dict = 1.00 all = 3.1855 } Zip/UnZip{ index = 0.0952 value = 0.3349 dict = 1.00 all = 0.3139 } ---------------------------- total value len = 62.526278 GB avg = 0.180 KB (by entry num) total key len = 6.840634 GB avg = 0.020 KB total ukey len = 3.226562 GB avg = 0.008 KB total ukey num = 0.391186279 Billion total entry num = 0.348064287 Billion write speed all = 2.540613717 MB/sec (with seq num) write speed all = 2.438628852 MB/sec (without seq num) 2024/08/20-01:09:04.442890 140500296816192 [compaction_job.cc:1944] [default] [JOB 4063] Generated table #14624246: 463429 keys, 21067276 bytes, temperature: kWarm 2024/08/20-01:09:04.442965 140500296816192 EVENT_LOG_v1 {"time_micros": 1724087344442918, "cf_name": "default", "job": 4063, "event": "table_file_creation", "file_number": 14624246, "file_size": 21067276, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189636721738, "largest_seqno": 189684753226, "table_properties": {"data_size": 18127856, "index_size": 882688, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 12976012, "raw_average_key_size": 28, "raw_value_size": 54132918, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 463429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1724087341, "oldest_key_time": 0, "file_creation_time": 1724087341, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14624246, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-01:09:04.472642 140500296816192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624247.sst) /dev/shm/ToplingTemp/Topling-88878-rrvFTJ 2024/08/20-01:09:04.978800 140500296816192 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624247.sst): first pass time = 0.51's, 132.595'MB/sec 2024/08/20-01:09:05.937480 140500296816192 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624247.sst): old prealloc_size = 73819750, real_size = 19077040 2024/08/20-01:09:05.937829 140500296816192 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624247.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 462324 prefix = 4 raw-key = 0.0074 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0542 GB zip-val = 0.0181 GB avg-val = 117.16 avg-zval = 39.11 2024/08/20-01:09:05.938269 140500296816192 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624247.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.020 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.067 GB 2024/08/20-01:09:06.308686 140500296816192 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624247.sst): second pass time = 0.96's, 56.551'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.37's, 56.608'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2722.208'MB/sec (index lex order gen) rebuild zvType time = 0.30's, 0.382'MB/sec write SST data time = 0.07's, 307.944'MB/sec dict compress time = 0.00's, 545582.000'MB/sec z-dict build time = 0.00's, sample length = 0.546'MB, throughput = 14480.121'MB/sec zip my value time = 0.96's, unzip length = 0.054'GB zip my value throughput = 56.551'MB/sec zip pipeline throughput = 102.355'MB/sec entries = 462324 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.16 avg-zval = 39.11 usrkeys = 462324 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.16 avg-zval = 39.11 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 462324 bytes = 1502560 } seq expand size = 3236268 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0092 GB value = 0.0542 GB dict = 0.55 MB all = 0.0671 GB } __ZipSize{ index = 0.0009 GB value = 0.0181 GB dict = 0.55 MB all = 0.0210 GB } UnZip/Zip{ index = 10.5043 value = 2.9956 dict = 1.00 all = 3.1937 } Zip/UnZip{ index = 0.0952 value = 0.3338 dict = 1.00 all = 0.3131 } ---------------------------- total value len = 62.583679 GB avg = 0.180 KB (by entry num) total key len = 6.853579 GB avg = 0.020 KB total ukey len = 3.235808 GB avg = 0.008 KB total ukey num = 0.391648603 Billion total entry num = 0.348526611 Billion write speed all = 2.543013637 MB/sec (with seq num) write speed all = 2.440900396 MB/sec (without seq num) 2024/08/20-01:09:06.337450 140500296816192 [compaction_job.cc:1944] [default] [JOB 4063] Generated table #14624247: 462324 keys, 21013118 bytes, temperature: kWarm 2024/08/20-01:09:06.337519 140500296816192 EVENT_LOG_v1 {"time_micros": 1724087346337480, "cf_name": "default", "job": 4063, "event": "table_file_creation", "file_number": 14624247, "file_size": 21013118, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189636722107, "largest_seqno": 189684752914, "table_properties": {"data_size": 18080992, "index_size": 880256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 12945072, "raw_average_key_size": 28, "raw_value_size": 54163880, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 462324, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1724087344, "oldest_key_time": 0, "file_creation_time": 1724087344, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14624247, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-01:09:06.365191 140500296816192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624248.sst) /dev/shm/ToplingTemp/Topling-88878-PilcPI 2024/08/20-01:09:06.779289 140500296816192 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624248.sst): first pass time = 0.41's, 162.074'MB/sec 2024/08/20-01:09:07.487098 140500296816192 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624248.sst): old prealloc_size = 73819750, real_size = 19185808 2024/08/20-01:09:07.487439 140500296816192 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624248.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 463278 prefix = 4 raw-key = 0.0074 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0541 GB zip-val = 0.0182 GB avg-val = 116.86 avg-zval = 39.26 2024/08/20-01:09:07.487879 140500296816192 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624248.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.020 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.067 GB 2024/08/20-01:09:07.856524 140500296816192 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624248.sst): second pass time = 0.71's, 76.574'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.37's, 57.182'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2815.931'MB/sec (index lex order gen) rebuild zvType time = 0.30's, 0.383'MB/sec write SST data time = 0.07's, 315.312'MB/sec dict compress time = 0.00's, 544970.000'MB/sec z-dict build time = 0.00's, sample length = 0.545'MB, throughput = 14511.250'MB/sec zip my value time = 0.71's, unzip length = 0.054'GB zip my value throughput = 76.574'MB/sec zip pipeline throughput = 76.574'MB/sec entries = 463278 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.86 avg-zval = 39.26 usrkeys = 463278 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.86 avg-zval = 39.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 463278 bytes = 1505664 } seq expand size = 3242946 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0093 GB value = 0.0541 GB dict = 0.54 MB all = 0.0671 GB } __ZipSize{ index = 0.0009 GB value = 0.0182 GB dict = 0.54 MB all = 0.0211 GB } UnZip/Zip{ index = 10.4993 value = 2.9766 dict = 1.00 all = 3.1769 } Zip/UnZip{ index = 0.0952 value = 0.3359 dict = 1.00 all = 0.3148 } ---------------------------- total value len = 62.641059 GB avg = 0.179 KB (by entry num) total key len = 6.866551 GB avg = 0.020 KB total ukey len = 3.245074 GB avg = 0.008 KB total ukey num = 0.392111881 Billion total entry num = 0.348989889 Billion write speed all = 2.545445853 MB/sec (with seq num) write speed all = 2.443202674 MB/sec (without seq num) 2024/08/20-01:09:07.886030 140500296816192 [compaction_job.cc:1944] [default] [JOB 4063] Generated table #14624248: 463278 keys, 21124186 bytes, temperature: kWarm 2024/08/20-01:09:07.886097 140500296816192 EVENT_LOG_v1 {"time_micros": 1724087347886062, "cf_name": "default", "job": 4063, "event": "table_file_creation", "file_number": 14624248, "file_size": 21124186, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189636722146, "largest_seqno": 189684753101, "table_properties": {"data_size": 18187328, "index_size": 882496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 12971784, "raw_average_key_size": 28, "raw_value_size": 54137116, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 463278, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1724087346, "oldest_key_time": 0, "file_creation_time": 1724087346, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14624248, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-01:09:07.928173 140500296816192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624249.sst) /dev/shm/ToplingTemp/Topling-88878-6Ih5WH 2024/08/20-01:09:07.996161 140500296816192 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624249.sst): first pass time = 0.07's, 261.129'MB/sec 2024/08/20-01:09:08.479812 140500296816192 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624249.sst): old prealloc_size = 73819750, real_size = 5385040 2024/08/20-01:09:08.479988 140500296816192 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624249.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 122511 prefix = 4 raw-key = 0.0020 GB zip-key = 0.0002 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0143 GB zip-val = 0.0051 GB avg-val = 116.82 avg-zval = 41.78 2024/08/20-01:09:08.480149 140500296816192 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624249.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.020 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.018 GB 2024/08/20-01:09:08.623084 140500296816192 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00/14624249.sst): second pass time = 0.48's, 29.645'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.14's, 41.202'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1603.405'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.252'MB/sec write SST data time = 0.02's, 274.509'MB/sec dict compress time = 0.00's, 149850.000'MB/sec z-dict build time = 0.00's, sample length = 0.150'MB, throughput = 4723.107'MB/sec zip my value time = 0.48's, unzip length = 0.014'GB zip my value throughput = 29.645'MB/sec zip pipeline throughput = 148.752'MB/sec entries = 122511 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.82 avg-zval = 41.78 usrkeys = 122511 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.82 avg-zval = 41.78 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 122511 bytes = 398176 } seq expand size = 857577 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0025 GB value = 0.0143 GB dict = 0.15 MB all = 0.0177 GB } __ZipSize{ index = 0.0002 GB value = 0.0051 GB dict = 0.15 MB all = 0.0059 GB } UnZip/Zip{ index = 10.3893 value = 2.7962 dict = 1.00 all = 3.0050 } Zip/UnZip{ index = 0.0963 value = 0.3576 dict = 1.00 all = 0.3328 } ---------------------------- total value len = 62.656228 GB avg = 0.179 KB (by entry num) total key len = 6.869981 GB avg = 0.020 KB total ukey len = 3.247524 GB avg = 0.008 KB total ukey num = 0.392234392 Billion total entry num = 0.349112400 Billion write speed all = 2.546055528 MB/sec (with seq num) write speed all = 2.443779329 MB/sec (without seq num) 2024/08/20-01:09:08.638947 140500296816192 [compaction_job.cc:1944] [default] [JOB 4063] Generated table #14624249: 122511 keys, 5904234 bytes, temperature: kWarm 2024/08/20-01:09:08.639014 140500296816192 EVENT_LOG_v1 {"time_micros": 1724087348638981, "cf_name": "default", "job": 4063, "event": "table_file_creation", "file_number": 14624249, "file_size": 5904234, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189640135622, "largest_seqno": 189681330552, "table_properties": {"data_size": 5118400, "index_size": 235840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3430308, "raw_average_key_size": 28, "raw_value_size": 14312046, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 122511, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1724087347, "oldest_key_time": 0, "file_creation_time": 1724087347, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14624249, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-01:09:08.646558 140500296816192 [compaction_job.cc:699] job-04063: subcompact[0], size: 0.132441 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04063/att-00 [14624243,14624244,14624245,14624246,14624247,14624248,14624249] 2024/08/20-01:09:08.737937 140500296816192 (Original Log Time 2024/08/20-01:08:46.994493) EVENT_LOG_v1 {"time_micros": 1724087326994446, "job": 4063, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L1": [14624234], "files_L2": [14624193, 14624194, 14624195, 14624196, 14624197, 14624198], "score": 0.815312, "input_data_size": 160024655, "oldest_snapshot_seqno": 189684786361} 2024/08/20-01:09:08.737942 140500296816192 (Original Log Time 2024/08/20-01:09:08.727765) [compaction_job.cc:2018] [default] [JOB 4063] Compacted 1@1 + 6@2 files to L2 => 132441374 bytes 2024/08/20-01:09:08.737943 140500296816192 (Original Log Time 2024/08/20-01:09:08.737850) [compaction_job.cc:1220] [default] compacted to: files[0 0 7 0 0 0 0] max score 0.05, MB/sec: 7.4 rd, 6.1 wr, level 2, files in(1, 6) out(7 +0 blob) MB in(31.8, 120.8 +0.0 blob) out(126.3 +0.0 blob), read-write-amplify(8.8) write-amplify(4.0) OK, records in: 3011779, records dropped: 110948 output_compression: Snappy 2024/08/20-01:09:08.737952 140500296816192 (Original Log Time 2024/08/20-01:09:08.737902) EVENT_LOG_v1 {"time_micros": 1724087348737878, "cf": "default", "job": 4063, "event": "compaction_finished", "compaction_time_micros": 21652077, "compaction_time_cpu_micros": 5393808, "output_level": 2, "num_output_files": 7, "total_output_size": 132441374, "num_input_records": 3011779, "num_output_records": 2900831, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 119721876, "file_range_sync_nanos": 0, "file_fsync_nanos": 6064, "file_prepare_write_nanos": 18353, "lsm_state": [0, 0, 7, 0, 0, 0, 0]} 2024/08/20-01:09:08.953373 140500296816192 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"a007a635-5c0a-4b4a-be96-b9c695cb016b","dbStarts":1724082090,"starts":1724087326,"executesMs":21829,"compactionJobId":4063,"attempt":0,"compactionInputRawBytes":436150618,"compactionInputZipBytes":160024655,"compactionOutputRawBytes":0,"compactionOutputZipBytes":132441374,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-0b539e80acf54f4efd9a0cf225affda9-466f6e1c2f101723-00"}}, response = 2024/08/20-01:09:08.953471 140500296816192 INFO dcompact_worker.cpp:1456: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-08-19T23.41.30/.rocksdb/job-04063/att-00: olev 2, work 21.829 s, result 22.975 ms, install 10.228 ms, input{raw 415.946 MiB zip 152.611 MiB}