2024/08/21-03:00:51.444927 140539937289792 DEBG dcompact_worker.cpp:2211: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00: fork to child time = 0.032948 sec 2024/08/21-03:00:51.445439 140539937289792 DEBG dcompact_worker.cpp:1011: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00 2024/08/21-03:00:51.449234 140539937289792 DEBG dcompact_worker.cpp:1031: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00 2024/08/21-03:00:51.450127 140539937289792 TRAC dcompact_worker.cpp:1081: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/08/21-03:00:51.450352 140539937289792 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/08/21-03:00:51.486232 140539937289792 [version_set.cc:6139] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-08-20T02.42.00/.rocksdb/job-62611/att-00/MANIFEST-14629260 2024/08/21-03:00:51.503269 140539937289792 [version_set.cc:6182] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-08-20T02.42.00/.rocksdb/job-62611/att-00/MANIFEST-14629260 succeeded,manifest_file_number is 14629260, next_file_number is 14701345, last_sequence is 191368746266, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 14701310 2024/08/21-03:00:51.503276 140539937289792 [version_set.cc:6197] Column family [default] (ID 0), log number is 0 2024/08/21-03:00:51.503334 140539937289792 [version_set.cc:5652] Creating manifest 14701345 2024/08/21-03:00:51.533774 140539937289792 DEBG dcompact_worker.cpp:1215: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00: bottommost_level: fake = 1, rpc = 0 2024/08/21-03:00:51.560077 140539937289792 [compaction_job.cc:2433] [default] [JOB 62611] Compacting 1@1 + 6@2 files to L2, score 1.00, subcompactions 1 : 1 2024/08/21-03:00:51.560098 140539937289792 [compaction_job.cc:2441] [default]: Compaction start summary: Base version 31523 Base level 1, inputs: [14701327(31MB)], [14701277(20MB) 14701278(20MB) 14701279(20MB) 14701280(20MB) 14701281(19MB) 14701295(20MB)] 2024/08/21-03:00:51.826221 140539937289792 [top_zip_table.cc:59] core git_version_hash_info_is:commit 7517bad5c1a147474c3830831b9a22c996321ae8 Author: leipeng Date: 2024-01-08 19:18:17 +0800 vfork_cmd: call close before wait_finish g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/08/21-03:00:51.826254 140539937289792 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit 7517bad5c1a147474c3830831b9a22c996321ae8 Author: leipeng Date: 2024-01-08 19:18:17 +0800 vfork_cmd: call close before wait_finish g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/08/21-03:00:51.826268 140539937289792 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit 7517bad5c1a147474c3830831b9a22c996321ae8 Author: leipeng Date: 2024-01-08 19:18:17 +0800 vfork_cmd: call close before wait_finish g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/08/21-03:00:51.826277 140539937289792 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit 7f80399bfbd2071b6e0f5683e8ab7225b8e6e394 Author: leipeng Date: 2024-01-19 14:52:30 +0800 Change ToplingZipTableOptions::sampleRatio to vector g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/08/21-03:00:51.826286 140539937289792 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit 1240987ca13d0fe562c9b9c037155079f0877840 Author: leipeng Date: 2024-01-19 15:07:04 +0800 Change ToplingZipTableOptions::sampleRatio to vector g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/08/21-03:00:52.005258 140539937289792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701346.sst) /dev/shm/ToplingTemp/Topling-32870-GlRHDE 2024/08/21-03:00:52.382784 140539937289792 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701346.sst): first pass time = 0.38's, 177.784'MB/sec 2024/08/21-03:00:54.127084 140539937289792 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701346.sst): old prealloc_size = 73819750, real_size = 19901264 2024/08/21-03:00:54.127480 140539937289792 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701346.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 466342 prefix = 4 raw-key = 0.0075 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.0541 GB zip-val = 0.0189 GB avg-val = 115.90 avg-zval = 40.53 2024/08/21-03:00:54.127949 140539937289792 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701346.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.067 GB 2024/08/21-03:00:54.945997 140539937289792 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701346.sst): second pass time = 1.73's, 31.325'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 26.538'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2256.107'MB/sec (index lex order gen) rebuild zvType time = 0.65's, 0.178'MB/sec write SST data time = 0.16's, 132.307'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.543'MB, throughput = 15341.734'MB/sec zip my value time = 1.73's, unzip length = 0.054'GB zip my value throughput = 31.325'MB/sec zip pipeline throughput = 90.088'MB/sec entries = 466342 avg-key = 28.00 avg-zkey = 1.89 avg-val = 115.90 avg-zval = 40.53 usrkeys = 466342 avg-key = 20.00 avg-zkey = 1.89 avg-val = 122.90 avg-zval = 40.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 466342 bytes = 1399040 } seq expand size = 3264394 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0093 GB value = 0.0541 GB dict = 0.54 MB all = 0.0671 GB } __ZipSize{ index = 0.0009 GB value = 0.0189 GB dict = 0.54 MB all = 0.0217 GB } UnZip/Zip{ index = 10.5725 value = 2.8595 dict = 1.00 all = 3.0880 } Zip/UnZip{ index = 0.0946 value = 0.3497 dict = 1.00 all = 0.3238 } ---------------------------- total value len = 0.057316 GB avg = 0.123 KB (by entry num) total key len = 0.013058 GB avg = 0.028 KB total ukey len = 0.009327 GB avg = 0.020 KB total ukey num = 0.000466342 Billion total entry num = 0.000466342 Billion write speed all = 0.011829747 MB/sec (with seq num) write speed all = 0.011202611 MB/sec (without seq num) 2024/08/21-03:00:54.978224 140539937289792 [compaction_job.cc:1944] [default] [JOB 62611] Generated table #14701346: 466342 keys, 21732126 bytes, temperature: kWarm 2024/08/21-03:00:54.978337 140539937289792 EVENT_LOG_v1 {"time_micros": 1724180454978274, "cf_name": "default", "job": 62611, "event": "table_file_creation", "file_number": 14701346, "file_size": 21732126, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 191353327179, "largest_seqno": 191368731374, "table_properties": {"data_size": 18902336, "index_size": 882176, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 13057576, "raw_average_key_size": 28, "raw_value_size": 54051356, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 466342, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1724180451, "oldest_key_time": 0, "file_creation_time": 1724180451, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "90ZT1ZM5QNL093I8HHQG", "orig_file_number": 14701346, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/21-03:00:55.013084 140539937289792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701347.sst) /dev/shm/ToplingTemp/Topling-32870-ayXETH 2024/08/21-03:00:55.783973 140539937289792 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701347.sst): first pass time = 0.77's, 87.059'MB/sec 2024/08/21-03:00:59.570230 140539937289792 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701347.sst): old prealloc_size = 73819750, real_size = 19352912 2024/08/21-03:00:59.570577 140539937289792 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701347.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 466855 prefix = 4 raw-key = 0.0075 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0540 GB zip-val = 0.0183 GB avg-val = 115.75 avg-zval = 39.30 2024/08/21-03:00:59.571027 140539937289792 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701347.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.067 GB 2024/08/21-03:01:00.059049 140539937289792 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701347.sst): second pass time = 3.79's, 14.276'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.49's, 44.056'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2798.874'MB/sec (index lex order gen) rebuild zvType time = 0.42's, 0.279'MB/sec write SST data time = 0.07's, 309.213'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.542'MB, throughput = 8322.423'MB/sec zip my value time = 3.79's, unzip length = 0.054'GB zip my value throughput = 14.276'MB/sec zip pipeline throughput = 178.581'MB/sec entries = 466855 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.75 avg-zval = 39.30 usrkeys = 466855 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.75 avg-zval = 39.30 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 466855 bytes = 1750720 } seq expand size = 3267985 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0093 GB value = 0.0540 GB dict = 0.54 MB all = 0.0671 GB } __ZipSize{ index = 0.0009 GB value = 0.0183 GB dict = 0.54 MB all = 0.0215 GB } UnZip/Zip{ index = 10.5011 value = 2.9453 dict = 1.00 all = 3.1163 } Zip/UnZip{ index = 0.0952 value = 0.3395 dict = 1.00 all = 0.3209 } ---------------------------- total value len = 0.114621 GB avg = 0.123 KB (by entry num) total key len = 0.026130 GB avg = 0.028 KB total ukey len = 0.018664 GB avg = 0.020 KB total ukey num = 0.000933197 Billion total entry num = 0.000933197 Billion write speed all = 0.023639784 MB/sec (with seq num) write speed all = 0.022385899 MB/sec (without seq num) 2024/08/21-03:01:00.090756 140539937289792 [compaction_job.cc:1944] [default] [JOB 62611] Generated table #14701347: 466855 keys, 21534700 bytes, temperature: kWarm 2024/08/21-03:01:00.090821 140539937289792 EVENT_LOG_v1 {"time_micros": 1724180460090787, "cf_name": "default", "job": 62611, "event": "table_file_creation", "file_number": 14701347, "file_size": 21534700, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 190586022308, "largest_seqno": 191368730869, "table_properties": {"data_size": 18346880, "index_size": 889152, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 13071940, "raw_average_key_size": 28, "raw_value_size": 54037018, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 466855, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1724180454, "oldest_key_time": 0, "file_creation_time": 1724180454, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "90ZT1ZM5QNL093I8HHQG", "orig_file_number": 14701347, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/21-03:01:00.181775 140539937289792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701348.sst) /dev/shm/ToplingTemp/Topling-32870-nyiGXF 2024/08/21-03:01:00.878980 140539937289792 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701348.sst): first pass time = 0.70's, 96.260'MB/sec 2024/08/21-03:01:02.178658 140539937289792 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701348.sst): old prealloc_size = 73819750, real_size = 19495904 2024/08/21-03:01:02.178958 140539937289792 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701348.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 466201 prefix = 4 raw-key = 0.0075 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.0541 GB zip-val = 0.0185 GB avg-val = 115.95 avg-zval = 39.68 2024/08/21-03:01:02.179389 140539937289792 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701348.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.067 GB 2024/08/21-03:01:02.699150 140539937289792 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701348.sst): second pass time = 1.30's, 41.659'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.52's, 41.200'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3160.916'MB/sec (index lex order gen) rebuild zvType time = 0.40's, 0.291'MB/sec write SST data time = 0.12's, 178.877'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.543'MB, throughput = 17687.958'MB/sec zip my value time = 1.30's, unzip length = 0.054'GB zip my value throughput = 41.659'MB/sec zip pipeline throughput = 86.677'MB/sec entries = 466201 avg-key = 28.00 avg-zkey = 1.89 avg-val = 115.95 avg-zval = 39.68 usrkeys = 466201 avg-key = 20.00 avg-zkey = 1.89 avg-val = 122.95 avg-zval = 39.68 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 466201 bytes = 1515168 } seq expand size = 3263407 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0093 GB value = 0.0541 GB dict = 0.54 MB all = 0.0671 GB } __ZipSize{ index = 0.0009 GB value = 0.0185 GB dict = 0.54 MB all = 0.0214 GB } UnZip/Zip{ index = 10.5716 value = 2.9224 dict = 1.00 all = 3.1297 } Zip/UnZip{ index = 0.0946 value = 0.3422 dict = 1.00 all = 0.3195 } ---------------------------- total value len = 0.171940 GB avg = 0.123 KB (by entry num) total key len = 0.039183 GB avg = 0.028 KB total ukey len = 0.027988 GB avg = 0.020 KB total ukey num = 0.001399398 Billion total entry num = 0.001399398 Billion write speed all = 0.035443497 MB/sec (with seq num) write speed all = 0.033564038 MB/sec (without seq num) 2024/08/21-03:01:02.729854 140539937289792 [compaction_job.cc:1944] [default] [JOB 62611] Generated table #14701348: 466201 keys, 21442894 bytes, temperature: kWarm 2024/08/21-03:01:02.729941 140539937289792 EVENT_LOG_v1 {"time_micros": 1724180462729893, "cf_name": "default", "job": 62611, "event": "table_file_creation", "file_number": 14701348, "file_size": 21442894, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 191319945619, "largest_seqno": 191368730354, "table_properties": {"data_size": 18497168, "index_size": 881984, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 13053628, "raw_average_key_size": 28, "raw_value_size": 54055346, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 466201, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1724180460, "oldest_key_time": 0, "file_creation_time": 1724180460, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "90ZT1ZM5QNL093I8HHQG", "orig_file_number": 14701348, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/21-03:01:02.767534 140539937289792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701349.sst) /dev/shm/ToplingTemp/Topling-32870-4uQgLF 2024/08/21-03:01:03.383981 140539937289792 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701349.sst): first pass time = 0.62's, 108.874'MB/sec 2024/08/21-03:01:04.094563 140539937289792 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701349.sst): old prealloc_size = 73819750, real_size = 19151920 2024/08/21-03:01:04.094869 140539937289792 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701349.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 466112 prefix = 4 raw-key = 0.0075 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0541 GB zip-val = 0.0181 GB avg-val = 115.98 avg-zval = 38.93 2024/08/21-03:01:04.095677 140539937289792 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701349.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.067 GB 2024/08/21-03:01:04.767754 140539937289792 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701349.sst): second pass time = 0.71's, 76.158'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.67's, 31.341'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2912.460'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.248'MB/sec write SST data time = 0.20's, 103.825'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.543'MB, throughput = 17941.402'MB/sec zip my value time = 0.71's, unzip length = 0.054'GB zip my value throughput = 76.158'MB/sec zip pipeline throughput = 76.158'MB/sec entries = 466112 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.98 avg-zval = 38.93 usrkeys = 466112 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.98 avg-zval = 38.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 466112 bytes = 1514880 } seq expand size = 3262784 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0093 GB value = 0.0541 GB dict = 0.54 MB all = 0.0671 GB } __ZipSize{ index = 0.0009 GB value = 0.0181 GB dict = 0.54 MB all = 0.0211 GB } UnZip/Zip{ index = 10.5018 value = 2.9788 dict = 1.00 all = 3.1810 } Zip/UnZip{ index = 0.0952 value = 0.3357 dict = 1.00 all = 0.3144 } ---------------------------- total value len = 0.229260 GB avg = 0.123 KB (by entry num) total key len = 0.052234 GB avg = 0.028 KB total ukey len = 0.037310 GB avg = 0.020 KB total ukey num = 0.001865510 Billion total entry num = 0.001865510 Billion write speed all = 0.047241176 MB/sec (with seq num) write speed all = 0.044736575 MB/sec (without seq num) 2024/08/21-03:01:04.797150 140539937289792 [compaction_job.cc:1944] [default] [JOB 62611] Generated table #14701349: 466112 keys, 21097030 bytes, temperature: kWarm 2024/08/21-03:01:04.797216 140539937289792 EVENT_LOG_v1 {"time_micros": 1724180464797182, "cf_name": "default", "job": 62611, "event": "table_file_creation", "file_number": 14701349, "file_size": 21097030, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 191306012714, "largest_seqno": 191368730185, "table_properties": {"data_size": 18147552, "index_size": 887680, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 13051136, "raw_average_key_size": 28, "raw_value_size": 54057828, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 466112, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1724180462, "oldest_key_time": 0, "file_creation_time": 1724180462, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "90ZT1ZM5QNL093I8HHQG", "orig_file_number": 14701349, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/21-03:01:04.841922 140539937289792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701350.sst) /dev/shm/ToplingTemp/Topling-32870-PP0oPE 2024/08/21-03:01:05.271853 140539937289792 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701350.sst): first pass time = 0.43's, 156.107'MB/sec 2024/08/21-03:01:06.039899 140539937289792 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701350.sst): old prealloc_size = 73819750, real_size = 19363328 2024/08/21-03:01:06.040266 140539937289792 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701350.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 466111 prefix = 4 raw-key = 0.0075 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.0541 GB zip-val = 0.0184 GB avg-val = 115.98 avg-zval = 39.40 2024/08/21-03:01:06.040744 140539937289792 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701350.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.067 GB 2024/08/21-03:01:06.774703 140539937289792 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701350.sst): second pass time = 0.77's, 70.470'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.73's, 29.082'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2564.132'MB/sec (index lex order gen) rebuild zvType time = 0.56's, 0.208'MB/sec write SST data time = 0.18's, 121.887'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.543'MB, throughput = 20215.977'MB/sec zip my value time = 0.77's, unzip length = 0.054'GB zip my value throughput = 70.470'MB/sec zip pipeline throughput = 70.470'MB/sec entries = 466111 avg-key = 28.00 avg-zkey = 1.89 avg-val = 115.98 avg-zval = 39.40 usrkeys = 466111 avg-key = 20.00 avg-zkey = 1.89 avg-val = 122.98 avg-zval = 39.40 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 466111 bytes = 1573136 } seq expand size = 3262777 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0093 GB value = 0.0541 GB dict = 0.54 MB all = 0.0671 GB } __ZipSize{ index = 0.0009 GB value = 0.0184 GB dict = 0.54 MB all = 0.0214 GB } UnZip/Zip{ index = 10.5727 value = 2.9435 dict = 1.00 all = 3.1406 } Zip/UnZip{ index = 0.0946 value = 0.3397 dict = 1.00 all = 0.3184 } ---------------------------- total value len = 0.286581 GB avg = 0.123 KB (by entry num) total key len = 0.065285 GB avg = 0.028 KB total ukey len = 0.046632 GB avg = 0.020 KB total ukey num = 0.002331621 Billion total entry num = 0.002331621 Billion write speed all = 0.059031260 MB/sec (with seq num) write speed all = 0.055901920 MB/sec (without seq num) 2024/08/21-03:01:06.806070 140539937289792 [compaction_job.cc:1944] [default] [JOB 62611] Generated table #14701350: 466111 keys, 21368442 bytes, temperature: kWarm 2024/08/21-03:01:06.806153 140539937289792 EVENT_LOG_v1 {"time_micros": 1724180466806106, "cf_name": "default", "job": 62611, "event": "table_file_creation", "file_number": 14701350, "file_size": 21368442, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 191279770607, "largest_seqno": 191368730694, "table_properties": {"data_size": 18364912, "index_size": 881728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 13051108, "raw_average_key_size": 28, "raw_value_size": 54057762, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 466111, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1724180464, "oldest_key_time": 0, "file_creation_time": 1724180464, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "90ZT1ZM5QNL093I8HHQG", "orig_file_number": 14701350, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/21-03:01:06.845000 140539937289792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701351.sst) /dev/shm/ToplingTemp/Topling-32870-oHVvUE 2024/08/21-03:01:07.329057 140539937289792 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701351.sst): first pass time = 0.48's, 138.654'MB/sec 2024/08/21-03:01:08.057614 140539937289792 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701351.sst): old prealloc_size = 73819750, real_size = 18612064 2024/08/21-03:01:08.057918 140539937289792 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701351.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 466965 prefix = 4 raw-key = 0.0075 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0540 GB zip-val = 0.0176 GB avg-val = 115.71 avg-zval = 37.69 2024/08/21-03:01:08.058491 140539937289792 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701351.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.067 GB 2024/08/21-03:01:08.465089 140539937289792 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701351.sst): second pass time = 0.73's, 74.226'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.41's, 50.600'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3141.861'MB/sec (index lex order gen) rebuild zvType time = 0.34's, 0.340'MB/sec write SST data time = 0.06's, 323.081'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.542'MB, throughput = 19140.446'MB/sec zip my value time = 0.73's, unzip length = 0.054'GB zip my value throughput = 74.226'MB/sec zip pipeline throughput = 74.226'MB/sec entries = 466965 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.71 avg-zval = 37.69 usrkeys = 466965 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.71 avg-zval = 37.69 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 466965 bytes = 1576016 } seq expand size = 3268755 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0093 GB value = 0.0540 GB dict = 0.54 MB all = 0.0671 GB } __ZipSize{ index = 0.0009 GB value = 0.0176 GB dict = 0.54 MB all = 0.0206 GB } UnZip/Zip{ index = 10.4360 value = 3.0701 dict = 1.00 all = 3.2550 } Zip/UnZip{ index = 0.0958 value = 0.3257 dict = 1.00 all = 0.3072 } ---------------------------- total value len = 0.343883 GB avg = 0.123 KB (by entry num) total key len = 0.078360 GB avg = 0.028 KB total ukey len = 0.055972 GB avg = 0.020 KB total ukey num = 0.002798586 Billion total entry num = 0.002798586 Billion write speed all = 0.070818181 MB/sec (with seq num) write speed all = 0.067063179 MB/sec (without seq num) 2024/08/21-03:01:08.492659 140539937289792 [compaction_job.cc:1944] [default] [JOB 62611] Generated table #14701351: 466965 keys, 20616966 bytes, temperature: kWarm 2024/08/21-03:01:08.492719 140539937289792 EVENT_LOG_v1 {"time_micros": 1724180468492686, "cf_name": "default", "job": 62611, "event": "table_file_creation", "file_number": 14701351, "file_size": 20616966, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 191248629892, "largest_seqno": 191368730561, "table_properties": {"data_size": 17600208, "index_size": 894912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 13075020, "raw_average_key_size": 28, "raw_value_size": 54033922, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 466965, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1724180466, "oldest_key_time": 0, "file_creation_time": 1724180466, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "90ZT1ZM5QNL093I8HHQG", "orig_file_number": 14701351, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/21-03:01:08.551312 140539937289792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701352.sst) /dev/shm/ToplingTemp/Topling-32870-ctvbkE 2024/08/21-03:01:08.688293 140539937289792 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701352.sst): first pass time = 0.14's, 244.513'MB/sec 2024/08/21-03:01:09.008584 140539937289792 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701352.sst): old prealloc_size = 73819750, real_size = 9633040 2024/08/21-03:01:09.008759 140539937289792 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701352.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 233339 prefix = 4 raw-key = 0.0037 GB zip-key = 0.0004 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0270 GB zip-val = 0.0091 GB avg-val = 115.50 avg-zval = 39.11 2024/08/21-03:01:09.008982 140539937289792 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701352.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.033 GB 2024/08/21-03:01:09.212231 140539937289792 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00/14701352.sst): second pass time = 0.32's, 84.333'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.20's, 52.265'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2883.599'MB/sec (index lex order gen) rebuild zvType time = 0.17's, 0.343'MB/sec write SST data time = 0.03's, 318.610'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.277'MB, throughput = 6151.863'MB/sec zip my value time = 0.32's, unzip length = 0.027'GB zip my value throughput = 84.333'MB/sec zip pipeline throughput = 84.333'MB/sec entries = 233339 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.50 avg-zval = 39.11 usrkeys = 233339 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.50 avg-zval = 39.11 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 233339 bytes = 787536 } seq expand size = 1633373 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0047 GB value = 0.0270 GB dict = 0.28 MB all = 0.0335 GB } __ZipSize{ index = 0.0004 GB value = 0.0091 GB dict = 0.28 MB all = 0.0106 GB } UnZip/Zip{ index = 10.4199 value = 2.9530 dict = 1.00 all = 3.1463 } Zip/UnZip{ index = 0.0960 value = 0.3386 dict = 1.00 all = 0.3178 } ---------------------------- total value len = 0.372468 GB avg = 0.123 KB (by entry num) total key len = 0.084894 GB avg = 0.028 KB total ukey len = 0.060638 GB avg = 0.020 KB total ukey num = 0.003031925 Billion total entry num = 0.003031925 Billion write speed all = 0.076698536 MB/sec (with seq num) write speed all = 0.072630961 MB/sec (without seq num) 2024/08/21-03:01:09.232300 140539937289792 [compaction_job.cc:1944] [default] [JOB 62611] Generated table #14701352: 233339 keys, 10642488 bytes, temperature: kWarm 2024/08/21-03:01:09.232350 140539937289792 EVENT_LOG_v1 {"time_micros": 1724180469232323, "cf_name": "default", "job": 62611, "event": "table_file_creation", "file_number": 14701352, "file_size": 10642488, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 191255597972, "largest_seqno": 191366982767, "table_properties": {"data_size": 9126656, "index_size": 447872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 6533492, "raw_average_key_size": 28, "raw_value_size": 26951254, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 233339, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1724180468, "oldest_key_time": 0, "file_creation_time": 1724180468, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "90ZT1ZM5QNL093I8HHQG", "orig_file_number": 14701352, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/21-03:01:09.244988 140539937289792 [compaction_job.cc:699] job-62611: subcompact[0], size: 0.138435 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-62611/att-00 [14701346,14701347,14701348,14701349,14701350,14701351,14701352] 2024/08/21-03:01:09.281816 140539937289792 (Original Log Time 2024/08/21-03:00:51.560156) EVENT_LOG_v1 {"time_micros": 1724180451560114, "job": 62611, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L1": [14701327], "files_L2": [14701277, 14701278, 14701279, 14701280, 14701281, 14701295], "score": 0.997126, "input_data_size": 161182189, "oldest_snapshot_seqno": 191368745814} 2024/08/21-03:01:09.281820 140539937289792 (Original Log Time 2024/08/21-03:01:09.278310) [compaction_job.cc:2018] [default] [JOB 62611] Compacted 1@1 + 6@2 files to L2 => 138434646 bytes 2024/08/21-03:01:09.281821 140539937289792 (Original Log Time 2024/08/21-03:01:09.281762) [compaction_job.cc:1220] [default] compacted to: files[0 0 7 0 0 0 0] max score 0.05, MB/sec: 9.1 rd, 7.8 wr, level 2, files in(1, 6) out(7 +0 blob) MB in(31.8, 121.9 +0.0 blob) out(132.0 +0.0 blob), read-write-amplify(9.0) write-amplify(4.2) OK, records in: 3031925, records dropped: 0 output_compression: Snappy 2024/08/21-03:01:09.281827 140539937289792 (Original Log Time 2024/08/21-03:01:09.281803) EVENT_LOG_v1 {"time_micros": 1724180469281784, "cf": "default", "job": 62611, "event": "compaction_finished", "compaction_time_micros": 17684835, "compaction_time_cpu_micros": 5840647, "output_level": 2, "num_output_files": 7, "total_output_size": 138434646, "num_input_records": 3031925, "num_output_records": 3031925, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 177544635, "file_range_sync_nanos": 0, "file_fsync_nanos": 6235, "file_prepare_write_nanos": 19173, "lsm_state": [0, 0, 7, 0, 0, 0, 0]} 2024/08/21-03:01:09.417836 140539937289792 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"a007a635-5c0a-4b4a-be96-b9c695cb016b","dbStarts":1724092920,"starts":1724180451,"executesMs":17826,"compactionJobId":62611,"attempt":0,"compactionInputRawBytes":436138386,"compactionInputZipBytes":161182189,"compactionOutputRawBytes":0,"compactionOutputZipBytes":138434646,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-fdb839884007ee6c8c149951b9d72a10-cb035f6cffebbf20-00"}}, response = 2024/08/21-03:01:09.417895 140539937289792 INFO dcompact_worker.cpp:1456: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-08-20T02.42.00/.rocksdb/job-62611/att-00: olev 2, work 17.827 s, result 6.347 ms, install 3.533 ms, input{raw 415.934 MiB zip 153.715 MiB}