2024/08/20-22:57:10.609689 139857135048256 DEBG dcompact_worker.cpp:2211: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00: fork to child time = 0.009859 sec 2024/08/20-22:57:10.610658 139857135048256 DEBG dcompact_worker.cpp:1011: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00 2024/08/20-22:57:10.615904 139857135048256 DEBG dcompact_worker.cpp:1031: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00 2024/08/20-22:57:10.617329 139857135048256 TRAC dcompact_worker.cpp:1081: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/08/20-22:57:10.617659 139857135048256 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/08/20-22:57:10.652214 139857135048256 [version_set.cc:6139] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-08-20T02.42.00/.rocksdb/job-54020/att-00/MANIFEST-14629260 2024/08/20-22:57:10.705697 139857135048256 [version_set.cc:6182] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-08-20T02.42.00/.rocksdb/job-54020/att-00/MANIFEST-14629260 succeeded,manifest_file_number is 14629260, next_file_number is 14691505, last_sequence is 191152461640, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 14691328 2024/08/20-22:57:10.705711 139857135048256 [version_set.cc:6197] Column family [default] (ID 0), log number is 0 2024/08/20-22:57:10.705814 139857135048256 [version_set.cc:5652] Creating manifest 14691505 2024/08/20-22:57:10.729981 139857135048256 DEBG dcompact_worker.cpp:1215: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00: bottommost_level: fake = 1, rpc = 0 2024/08/20-22:57:10.742248 139857135048256 [compaction_job.cc:2433] [default] [JOB 54020] Compacting 1@1 + 10@2 files to L2, score 0.99, subcompactions 1 : 1 2024/08/20-22:57:10.742281 139857135048256 [compaction_job.cc:2441] [default]: Compaction start summary: Base version 27346 Base level 1, inputs: [14691488(31MB)], [14691412(19MB) 14691413(20MB) 14691414(19MB) 14691415(19MB) 14691448(20MB) 14691449(20MB) 14691450(19MB) 14691451(19MB) 14691452(19MB) 14691453(19MB)] 2024/08/20-22:57:10.980672 139857135048256 [top_zip_table.cc:59] core git_version_hash_info_is:commit 7517bad5c1a147474c3830831b9a22c996321ae8 Author: leipeng Date: 2024-01-08 19:18:17 +0800 vfork_cmd: call close before wait_finish g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/08/20-22:57:10.980700 139857135048256 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit 7517bad5c1a147474c3830831b9a22c996321ae8 Author: leipeng Date: 2024-01-08 19:18:17 +0800 vfork_cmd: call close before wait_finish g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/08/20-22:57:10.980718 139857135048256 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit 7517bad5c1a147474c3830831b9a22c996321ae8 Author: leipeng Date: 2024-01-08 19:18:17 +0800 vfork_cmd: call close before wait_finish g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/08/20-22:57:10.980728 139857135048256 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit 7f80399bfbd2071b6e0f5683e8ab7225b8e6e394 Author: leipeng Date: 2024-01-19 14:52:30 +0800 Change ToplingZipTableOptions::sampleRatio to vector g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/08/20-22:57:10.980743 139857135048256 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit 1240987ca13d0fe562c9b9c037155079f0877840 Author: leipeng Date: 2024-01-19 15:07:04 +0800 Change ToplingZipTableOptions::sampleRatio to vector g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/08/20-22:57:11.131290 139857135048256 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691506.sst) /dev/shm/ToplingTemp/Topling-50207-gqpLhC 2024/08/20-22:57:11.383712 139857135048256 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691506.sst): first pass time = 0.25's, 265.917'MB/sec 2024/08/20-22:57:12.081009 139857135048256 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691506.sst): old prealloc_size = 73819750, real_size = 18534304 2024/08/20-22:57:12.081395 139857135048256 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691506.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 466764 prefix = 4 raw-key = 0.0075 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0540 GB zip-val = 0.0175 GB avg-val = 115.77 avg-zval = 37.55 2024/08/20-22:57:12.081846 139857135048256 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691506.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.067 GB 2024/08/20-22:57:12.457363 139857135048256 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691506.sst): second pass time = 0.70's, 77.605'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.38's, 54.573'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2466.409'MB/sec (index lex order gen) rebuild zvType time = 0.31's, 0.381'MB/sec write SST data time = 0.07's, 295.884'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.543'MB, throughput = 13119.505'MB/sec zip my value time = 0.70's, unzip length = 0.054'GB zip my value throughput = 77.605'MB/sec zip pipeline throughput = 77.605'MB/sec entries = 466764 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.77 avg-zval = 37.55 usrkeys = 466764 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.77 avg-zval = 37.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 466764 bytes = 1575344 } seq expand size = 3267348 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0093 GB value = 0.0540 GB dict = 0.54 MB all = 0.0671 GB } __ZipSize{ index = 0.0009 GB value = 0.0175 GB dict = 0.54 MB all = 0.0205 GB } UnZip/Zip{ index = 10.5044 value = 3.0829 dict = 1.00 all = 3.2674 } Zip/UnZip{ index = 0.0952 value = 0.3244 dict = 1.00 all = 0.3061 } ---------------------------- total value len = 0.057307 GB avg = 0.123 KB (by entry num) total key len = 0.013069 GB avg = 0.028 KB total ukey len = 0.009335 GB avg = 0.020 KB total ukey num = 0.000466764 Billion total entry num = 0.000466764 Billion write speed all = 0.007057124 MB/sec (with seq num) write speed all = 0.006682678 MB/sec (without seq num) 2024/08/20-22:57:12.485557 139857135048256 [compaction_job.cc:1944] [default] [JOB 54020] Generated table #14691506: 466764 keys, 20538976 bytes, temperature: kWarm 2024/08/20-22:57:12.485643 139857135048256 EVENT_LOG_v1 {"time_micros": 1724165832485595, "cf_name": "default", "job": 54020, "event": "table_file_creation", "file_number": 14691506, "file_size": 20538976, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 191073220730, "largest_seqno": 191152440997, "table_properties": {"data_size": 17528720, "index_size": 888704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 13069392, "raw_average_key_size": 28, "raw_value_size": 54039532, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 466764, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1724165831, "oldest_key_time": 0, "file_creation_time": 1724165831, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "90ZT1ZM5QNL093I8HHQG", "orig_file_number": 14691506, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-22:57:12.508113 139857135048256 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691507.sst) /dev/shm/ToplingTemp/Topling-50207-b8JZ1A 2024/08/20-22:57:12.910109 139857135048256 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691507.sst): first pass time = 0.40's, 166.958'MB/sec 2024/08/20-22:57:13.609953 139857135048256 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691507.sst): old prealloc_size = 73819750, real_size = 19020336 2024/08/20-22:57:13.610344 139857135048256 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691507.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 467010 prefix = 4 raw-key = 0.0075 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0540 GB zip-val = 0.0180 GB avg-val = 115.70 avg-zval = 38.57 2024/08/20-22:57:13.610764 139857135048256 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691507.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.067 GB 2024/08/20-22:57:13.980063 139857135048256 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691507.sst): second pass time = 0.70's, 77.302'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.37's, 56.815'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2468.179'MB/sec (index lex order gen) rebuild zvType time = 0.30's, 0.387'MB/sec write SST data time = 0.07's, 309.569'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.542'MB, throughput = 14235.513'MB/sec zip my value time = 0.70's, unzip length = 0.054'GB zip my value throughput = 77.302'MB/sec zip pipeline throughput = 77.302'MB/sec entries = 467010 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.70 avg-zval = 38.57 usrkeys = 467010 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.70 avg-zval = 38.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 467010 bytes = 1576176 } seq expand size = 3269070 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0093 GB value = 0.0540 GB dict = 0.54 MB all = 0.0671 GB } __ZipSize{ index = 0.0009 GB value = 0.0180 GB dict = 0.54 MB all = 0.0210 GB } UnZip/Zip{ index = 10.5031 value = 2.9995 dict = 1.00 all = 3.1915 } Zip/UnZip{ index = 0.0952 value = 0.3334 dict = 1.00 all = 0.3133 } ---------------------------- total value len = 0.114609 GB avg = 0.123 KB (by entry num) total key len = 0.026146 GB avg = 0.028 KB total ukey len = 0.018675 GB avg = 0.020 KB total ukey num = 0.000933774 Billion total entry num = 0.000933774 Billion write speed all = 0.014112270 MB/sec (with seq num) write speed all = 0.013363296 MB/sec (without seq num) 2024/08/20-22:57:14.010753 139857135048256 [compaction_job.cc:1944] [default] [JOB 54020] Generated table #14691507: 467010 keys, 21027502 bytes, temperature: kWarm 2024/08/20-22:57:14.010878 139857135048256 EVENT_LOG_v1 {"time_micros": 1724165834010790, "cf_name": "default", "job": 54020, "event": "table_file_creation", "file_number": 14691507, "file_size": 21027502, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 191052376133, "largest_seqno": 191152442417, "table_properties": {"data_size": 18014112, "index_size": 889280, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 13076280, "raw_average_key_size": 28, "raw_value_size": 54032684, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 467010, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1724165832, "oldest_key_time": 0, "file_creation_time": 1724165832, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "90ZT1ZM5QNL093I8HHQG", "orig_file_number": 14691507, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-22:57:14.034235 139857135048256 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691508.sst) /dev/shm/ToplingTemp/Topling-50207-kBA4IA 2024/08/20-22:57:14.435156 139857135048256 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691508.sst): first pass time = 0.40's, 167.414'MB/sec 2024/08/20-22:57:15.109338 139857135048256 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691508.sst): old prealloc_size = 73819750, real_size = 18516448 2024/08/20-22:57:15.109665 139857135048256 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691508.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 465861 prefix = 4 raw-key = 0.0075 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0541 GB zip-val = 0.0175 GB avg-val = 116.05 avg-zval = 37.58 2024/08/20-22:57:15.110087 139857135048256 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691508.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.067 GB 2024/08/20-22:57:15.484144 139857135048256 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691508.sst): second pass time = 0.67's, 80.293'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.37's, 54.903'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2913.106'MB/sec (index lex order gen) rebuild zvType time = 0.31's, 0.379'MB/sec write SST data time = 0.07's, 305.613'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.543'MB, throughput = 14518.174'MB/sec zip my value time = 0.67's, unzip length = 0.054'GB zip my value throughput = 80.293'MB/sec zip pipeline throughput = 80.293'MB/sec entries = 465861 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.05 avg-zval = 37.58 usrkeys = 465861 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.05 avg-zval = 37.58 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 465861 bytes = 1630528 } seq expand size = 3261027 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0093 GB value = 0.0541 GB dict = 0.54 MB all = 0.0671 GB } __ZipSize{ index = 0.0009 GB value = 0.0175 GB dict = 0.54 MB all = 0.0206 GB } UnZip/Zip{ index = 10.4360 value = 3.0882 dict = 1.00 all = 3.2613 } Zip/UnZip{ index = 0.0958 value = 0.3238 dict = 1.00 all = 0.3066 } ---------------------------- total value len = 0.171934 GB avg = 0.123 KB (by entry num) total key len = 0.039190 GB avg = 0.028 KB total ukey len = 0.027993 GB avg = 0.020 KB total ukey num = 0.001399635 Billion total entry num = 0.001399635 Billion write speed all = 0.021164493 MB/sec (with seq num) write speed all = 0.020042024 MB/sec (without seq num) 2024/08/20-22:57:15.512135 139857135048256 [compaction_job.cc:1944] [default] [JOB 54020] Generated table #14691508: 465861 keys, 20577340 bytes, temperature: kWarm 2024/08/20-22:57:15.512204 139857135048256 EVENT_LOG_v1 {"time_micros": 1724165835512171, "cf_name": "default", "job": 54020, "event": "table_file_creation", "file_number": 14691508, "file_size": 20577340, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 190996035480, "largest_seqno": 191152442725, "table_properties": {"data_size": 17507024, "index_size": 892800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 13044108, "raw_average_key_size": 28, "raw_value_size": 54064834, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 465861, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1724165834, "oldest_key_time": 0, "file_creation_time": 1724165834, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "90ZT1ZM5QNL093I8HHQG", "orig_file_number": 14691508, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-22:57:15.540608 139857135048256 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691509.sst) /dev/shm/ToplingTemp/Topling-50207-sODyKC 2024/08/20-22:57:15.943854 139857135048256 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691509.sst): first pass time = 0.40's, 166.442'MB/sec 2024/08/20-22:57:16.620346 139857135048256 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691509.sst): old prealloc_size = 73819750, real_size = 18920176 2024/08/20-22:57:16.620675 139857135048256 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691509.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 466097 prefix = 4 raw-key = 0.0075 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0541 GB zip-val = 0.0179 GB avg-val = 115.98 avg-zval = 38.44 2024/08/20-22:57:16.621117 139857135048256 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691509.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.067 GB 2024/08/20-22:57:16.991470 139857135048256 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691509.sst): second pass time = 0.68's, 80.009'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.37's, 56.222'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2956.453'MB/sec (index lex order gen) rebuild zvType time = 0.30's, 0.383'MB/sec write SST data time = 0.07's, 311.586'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.543'MB, throughput = 13874.333'MB/sec zip my value time = 0.68's, unzip length = 0.054'GB zip my value throughput = 80.009'MB/sec zip pipeline throughput = 80.009'MB/sec entries = 466097 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.98 avg-zval = 38.44 usrkeys = 466097 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.98 avg-zval = 38.44 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 466097 bytes = 1514832 } seq expand size = 3262679 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0093 GB value = 0.0541 GB dict = 0.54 MB all = 0.0671 GB } __ZipSize{ index = 0.0009 GB value = 0.0179 GB dict = 0.54 MB all = 0.0209 GB } UnZip/Zip{ index = 10.5022 value = 3.0173 dict = 1.00 all = 3.2163 } Zip/UnZip{ index = 0.0952 value = 0.3314 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 0.229255 GB avg = 0.123 KB (by entry num) total key len = 0.052240 GB avg = 0.028 KB total ukey len = 0.037315 GB avg = 0.020 KB total ukey num = 0.001865732 Billion total entry num = 0.001865732 Billion write speed all = 0.028214741 MB/sec (with seq num) write speed all = 0.026718701 MB/sec (without seq num) 2024/08/20-22:57:17.021291 139857135048256 [compaction_job.cc:1944] [default] [JOB 54020] Generated table #14691509: 466097 keys, 20865258 bytes, temperature: kWarm 2024/08/20-22:57:17.021361 139857135048256 EVENT_LOG_v1 {"time_micros": 1724165837021327, "cf_name": "default", "job": 54020, "event": "table_file_creation", "file_number": 14691509, "file_size": 20865258, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 191088553281, "largest_seqno": 191152443251, "table_properties": {"data_size": 17915872, "index_size": 887616, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 13050716, "raw_average_key_size": 28, "raw_value_size": 54058178, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 466097, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1724165835, "oldest_key_time": 0, "file_creation_time": 1724165835, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "90ZT1ZM5QNL093I8HHQG", "orig_file_number": 14691509, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-22:57:17.049595 139857135048256 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691510.sst) /dev/shm/ToplingTemp/Topling-50207-oYbfuE 2024/08/20-22:57:17.454006 139857135048256 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691510.sst): first pass time = 0.40's, 165.962'MB/sec 2024/08/20-22:57:18.148114 139857135048256 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691510.sst): old prealloc_size = 73819750, real_size = 18917088 2024/08/20-22:57:18.148481 139857135048256 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691510.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 466684 prefix = 4 raw-key = 0.0075 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0540 GB zip-val = 0.0179 GB avg-val = 115.80 avg-zval = 38.38 2024/08/20-22:57:18.148947 139857135048256 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691510.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.067 GB 2024/08/20-22:57:18.520068 139857135048256 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691510.sst): second pass time = 0.69's, 78.020'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.37's, 56.723'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2628.625'MB/sec (index lex order gen) rebuild zvType time = 0.30's, 0.384'MB/sec write SST data time = 0.07's, 311.295'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.543'MB, throughput = 13522.580'MB/sec zip my value time = 0.69's, unzip length = 0.054'GB zip my value throughput = 78.020'MB/sec zip pipeline throughput = 78.020'MB/sec entries = 466684 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.80 avg-zval = 38.38 usrkeys = 466684 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.80 avg-zval = 38.38 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 466684 bytes = 1750080 } seq expand size = 3266788 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0093 GB value = 0.0540 GB dict = 0.54 MB all = 0.0671 GB } __ZipSize{ index = 0.0009 GB value = 0.0179 GB dict = 0.54 MB all = 0.0211 GB } UnZip/Zip{ index = 10.5026 value = 3.0172 dict = 1.00 all = 3.1808 } Zip/UnZip{ index = 0.0952 value = 0.3314 dict = 1.00 all = 0.3144 } ---------------------------- total value len = 0.286564 GB avg = 0.123 KB (by entry num) total key len = 0.065308 GB avg = 0.028 KB total ukey len = 0.046648 GB avg = 0.020 KB total ukey num = 0.002332416 Billion total entry num = 0.002332416 Billion write speed all = 0.035263199 MB/sec (with seq num) write speed all = 0.033393233 MB/sec (without seq num) 2024/08/20-22:57:18.548902 139857135048256 [compaction_job.cc:1944] [default] [JOB 54020] Generated table #14691510: 466684 keys, 21098436 bytes, temperature: kWarm 2024/08/20-22:57:18.548985 139857135048256 EVENT_LOG_v1 {"time_micros": 1724165838548948, "cf_name": "default", "job": 54020, "event": "table_file_creation", "file_number": 14691510, "file_size": 21098436, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 190586022308, "largest_seqno": 191152443043, "table_properties": {"data_size": 17911504, "index_size": 888704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 13067152, "raw_average_key_size": 28, "raw_value_size": 54041764, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 466684, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1724165837, "oldest_key_time": 0, "file_creation_time": 1724165837, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "90ZT1ZM5QNL093I8HHQG", "orig_file_number": 14691510, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-22:57:18.570887 139857135048256 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691511.sst) /dev/shm/ToplingTemp/Topling-50207-dCyUqE 2024/08/20-22:57:18.972462 139857135048256 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691511.sst): first pass time = 0.40's, 167.137'MB/sec 2024/08/20-22:57:19.648045 139857135048256 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691511.sst): old prealloc_size = 73819750, real_size = 19251024 2024/08/20-22:57:19.648420 139857135048256 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691511.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 466618 prefix = 4 raw-key = 0.0075 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0540 GB zip-val = 0.0182 GB avg-val = 115.82 avg-zval = 39.09 2024/08/20-22:57:19.648863 139857135048256 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691511.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.067 GB 2024/08/20-22:57:20.014416 139857135048256 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691511.sst): second pass time = 0.67's, 80.091'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.37's, 58.016'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2569.582'MB/sec (index lex order gen) rebuild zvType time = 0.30's, 0.392'MB/sec write SST data time = 0.07's, 310.762'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.542'MB, throughput = 13775.989'MB/sec zip my value time = 0.67's, unzip length = 0.054'GB zip my value throughput = 80.091'MB/sec zip pipeline throughput = 80.091'MB/sec entries = 466618 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.82 avg-zval = 39.09 usrkeys = 466618 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.82 avg-zval = 39.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 466618 bytes = 1574848 } seq expand size = 3266326 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0093 GB value = 0.0540 GB dict = 0.54 MB all = 0.0671 GB } __ZipSize{ index = 0.0009 GB value = 0.0182 GB dict = 0.54 MB all = 0.0213 GB } UnZip/Zip{ index = 10.4327 value = 2.9630 dict = 1.00 all = 3.1573 } Zip/UnZip{ index = 0.0959 value = 0.3375 dict = 1.00 all = 0.3167 } ---------------------------- total value len = 0.343874 GB avg = 0.123 KB (by entry num) total key len = 0.078373 GB avg = 0.028 KB total ukey len = 0.055981 GB avg = 0.020 KB total ukey num = 0.002799034 Billion total entry num = 0.002799034 Billion write speed all = 0.042309597 MB/sec (with seq num) write speed all = 0.040065867 MB/sec (without seq num) 2024/08/20-22:57:20.042825 139857135048256 [compaction_job.cc:1944] [default] [JOB 54020] Generated table #14691511: 466618 keys, 21255186 bytes, temperature: kWarm 2024/08/20-22:57:20.042900 139857135048256 EVENT_LOG_v1 {"time_micros": 1724165840042863, "cf_name": "default", "job": 54020, "event": "table_file_creation", "file_number": 14691511, "file_size": 21255186, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 191037662625, "largest_seqno": 191152443092, "table_properties": {"data_size": 18239680, "index_size": 894528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 13065304, "raw_average_key_size": 28, "raw_value_size": 54043628, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 466618, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1724165838, "oldest_key_time": 0, "file_creation_time": 1724165838, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "90ZT1ZM5QNL093I8HHQG", "orig_file_number": 14691511, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-22:57:20.064235 139857135048256 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691512.sst) /dev/shm/ToplingTemp/Topling-50207-ofYubC 2024/08/20-22:57:20.466787 139857135048256 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691512.sst): first pass time = 0.40's, 166.745'MB/sec 2024/08/20-22:57:21.138735 139857135048256 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691512.sst): old prealloc_size = 73819750, real_size = 18955104 2024/08/20-22:57:21.139090 139857135048256 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691512.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 466584 prefix = 4 raw-key = 0.0075 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0540 GB zip-val = 0.0179 GB avg-val = 115.83 avg-zval = 38.47 2024/08/20-22:57:21.139560 139857135048256 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691512.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.067 GB 2024/08/20-22:57:21.509518 139857135048256 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691512.sst): second pass time = 0.67's, 80.534'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.37's, 56.374'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2633.501'MB/sec (index lex order gen) rebuild zvType time = 0.30's, 0.385'MB/sec write SST data time = 0.07's, 309.193'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.542'MB, throughput = 10490.259'MB/sec zip my value time = 0.67's, unzip length = 0.054'GB zip my value throughput = 80.534'MB/sec zip pipeline throughput = 80.534'MB/sec entries = 466584 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.83 avg-zval = 38.47 usrkeys = 466584 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.83 avg-zval = 38.47 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 466584 bytes = 1516416 } seq expand size = 3266088 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0093 GB value = 0.0540 GB dict = 0.54 MB all = 0.0671 GB } __ZipSize{ index = 0.0009 GB value = 0.0179 GB dict = 0.54 MB all = 0.0209 GB } UnZip/Zip{ index = 10.5003 value = 3.0109 dict = 1.00 all = 3.2106 } Zip/UnZip{ index = 0.0952 value = 0.3321 dict = 1.00 all = 0.3115 } ---------------------------- total value len = 0.401185 GB avg = 0.123 KB (by entry num) total key len = 0.091437 GB avg = 0.028 KB total ukey len = 0.065312 GB avg = 0.020 KB total ukey num = 0.003265618 Billion total entry num = 0.003265618 Billion write speed all = 0.049353862 MB/sec (with seq num) write speed all = 0.046736505 MB/sec (without seq num) 2024/08/20-22:57:21.538358 139857135048256 [compaction_job.cc:1944] [default] [JOB 54020] Generated table #14691512: 466584 keys, 20902482 bytes, temperature: kWarm 2024/08/20-22:57:21.538431 139857135048256 EVENT_LOG_v1 {"time_micros": 1724165841538395, "cf_name": "default", "job": 54020, "event": "table_file_creation", "file_number": 14691512, "file_size": 20902482, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 191102102124, "largest_seqno": 191152443539, "table_properties": {"data_size": 17949584, "index_size": 888704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 13064352, "raw_average_key_size": 28, "raw_value_size": 54044616, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 466584, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1724165840, "oldest_key_time": 0, "file_creation_time": 1724165840, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "90ZT1ZM5QNL093I8HHQG", "orig_file_number": 14691512, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-22:57:21.559685 139857135048256 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691513.sst) /dev/shm/ToplingTemp/Topling-50207-I6xUvC 2024/08/20-22:57:21.962546 139857135048256 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691513.sst): first pass time = 0.40's, 166.602'MB/sec 2024/08/20-22:57:22.628815 139857135048256 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691513.sst): old prealloc_size = 73819750, real_size = 18703504 2024/08/20-22:57:22.629174 139857135048256 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691513.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 466841 prefix = 4 raw-key = 0.0075 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0540 GB zip-val = 0.0177 GB avg-val = 115.75 avg-zval = 37.91 2024/08/20-22:57:22.629621 139857135048256 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691513.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.067 GB 2024/08/20-22:57:23.000643 139857135048256 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691513.sst): second pass time = 0.67's, 81.203'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.37's, 55.537'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2689.307'MB/sec (index lex order gen) rebuild zvType time = 0.30's, 0.384'MB/sec write SST data time = 0.07's, 305.415'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.542'MB, throughput = 12562.258'MB/sec zip my value time = 0.67's, unzip length = 0.054'GB zip my value throughput = 81.203'MB/sec zip pipeline throughput = 81.203'MB/sec entries = 466841 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.75 avg-zval = 37.91 usrkeys = 466841 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.75 avg-zval = 37.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 466841 bytes = 1517248 } seq expand size = 3267887 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0093 GB value = 0.0540 GB dict = 0.54 MB all = 0.0671 GB } __ZipSize{ index = 0.0009 GB value = 0.0177 GB dict = 0.54 MB all = 0.0207 GB } UnZip/Zip{ index = 10.5008 value = 3.0534 dict = 1.00 all = 3.2498 } Zip/UnZip{ index = 0.0952 value = 0.3275 dict = 1.00 all = 0.3077 } ---------------------------- total value len = 0.458490 GB avg = 0.123 KB (by entry num) total key len = 0.104509 GB avg = 0.028 KB total ukey len = 0.074649 GB avg = 0.020 KB total ukey num = 0.003732459 Billion total entry num = 0.003732459 Billion write speed all = 0.056396210 MB/sec (with seq num) write speed all = 0.053405133 MB/sec (without seq num) 2024/08/20-22:57:23.028873 139857135048256 [compaction_job.cc:1944] [default] [JOB 54020] Generated table #14691513: 466841 keys, 20650076 bytes, temperature: kWarm 2024/08/20-22:57:23.028947 139857135048256 EVENT_LOG_v1 {"time_micros": 1724165843028910, "cf_name": "default", "job": 54020, "event": "table_file_creation", "file_number": 14691513, "file_size": 20650076, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 191102102972, "largest_seqno": 191152442346, "table_properties": {"data_size": 17697472, "index_size": 889152, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 13071548, "raw_average_key_size": 28, "raw_value_size": 54037330, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 466841, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1724165841, "oldest_key_time": 0, "file_creation_time": 1724165841, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "90ZT1ZM5QNL093I8HHQG", "orig_file_number": 14691513, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-22:57:23.052174 139857135048256 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691514.sst) /dev/shm/ToplingTemp/Topling-50207-x8NVdE 2024/08/20-22:57:23.451521 139857135048256 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691514.sst): first pass time = 0.40's, 168.069'MB/sec 2024/08/20-22:57:24.138583 139857135048256 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691514.sst): old prealloc_size = 73819750, real_size = 19014560 2024/08/20-22:57:24.138924 139857135048256 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691514.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 465251 prefix = 4 raw-key = 0.0074 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0541 GB zip-val = 0.0180 GB avg-val = 116.24 avg-zval = 38.71 2024/08/20-22:57:24.139388 139857135048256 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691514.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.067 GB 2024/08/20-22:57:24.508039 139857135048256 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691514.sst): second pass time = 0.69's, 78.814'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.37's, 56.889'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2868.251'MB/sec (index lex order gen) rebuild zvType time = 0.30's, 0.384'MB/sec write SST data time = 0.07's, 317.248'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.544'MB, throughput = 12859.561'MB/sec zip my value time = 0.69's, unzip length = 0.054'GB zip my value throughput = 78.814'MB/sec zip pipeline throughput = 78.814'MB/sec entries = 465251 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.24 avg-zval = 38.71 usrkeys = 465251 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.24 avg-zval = 38.71 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 465251 bytes = 1570240 } seq expand size = 3256757 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0093 GB value = 0.0541 GB dict = 0.54 MB all = 0.0671 GB } __ZipSize{ index = 0.0009 GB value = 0.0180 GB dict = 0.54 MB all = 0.0210 GB } UnZip/Zip{ index = 10.5036 value = 3.0025 dict = 1.00 all = 3.1929 } Zip/UnZip{ index = 0.0952 value = 0.3331 dict = 1.00 all = 0.3132 } ---------------------------- total value len = 0.515828 GB avg = 0.123 KB (by entry num) total key len = 0.117536 GB avg = 0.028 KB total ukey len = 0.083954 GB avg = 0.020 KB total ukey num = 0.004197710 Billion total entry num = 0.004197710 Billion write speed all = 0.063435248 MB/sec (with seq num) write speed all = 0.060071841 MB/sec (without seq num) 2024/08/20-22:57:24.536301 139857135048256 [compaction_job.cc:1944] [default] [JOB 54020] Generated table #14691514: 465251 keys, 21017870 bytes, temperature: kWarm 2024/08/20-22:57:24.536380 139857135048256 EVENT_LOG_v1 {"time_micros": 1724165844536338, "cf_name": "default", "job": 54020, "event": "table_file_creation", "file_number": 14691514, "file_size": 21017870, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 191031976588, "largest_seqno": 191152442388, "table_properties": {"data_size": 18012176, "index_size": 885888, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 13027028, "raw_average_key_size": 28, "raw_value_size": 54081962, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 465251, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1724165843, "oldest_key_time": 0, "file_creation_time": 1724165843, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "90ZT1ZM5QNL093I8HHQG", "orig_file_number": 14691514, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-22:57:24.558879 139857135048256 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691515.sst) /dev/shm/ToplingTemp/Topling-50207-rhRPnE 2024/08/20-22:57:24.958593 139857135048256 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691515.sst): first pass time = 0.40's, 167.913'MB/sec 2024/08/20-22:57:25.649453 139857135048256 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691515.sst): old prealloc_size = 73819750, real_size = 18784400 2024/08/20-22:57:25.649787 139857135048256 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691515.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 466062 prefix = 4 raw-key = 0.0075 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0541 GB zip-val = 0.0178 GB avg-val = 115.99 avg-zval = 38.15 2024/08/20-22:57:25.650222 139857135048256 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691515.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.067 GB 2024/08/20-22:57:26.019803 139857135048256 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691515.sst): second pass time = 0.69's, 78.343'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.37's, 55.973'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2903.447'MB/sec (index lex order gen) rebuild zvType time = 0.30's, 0.384'MB/sec write SST data time = 0.07's, 311.968'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.543'MB, throughput = 10967.194'MB/sec zip my value time = 0.69's, unzip length = 0.054'GB zip my value throughput = 78.343'MB/sec zip pipeline throughput = 78.343'MB/sec entries = 466062 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.99 avg-zval = 38.15 usrkeys = 466062 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.99 avg-zval = 38.15 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 466062 bytes = 1514720 } seq expand size = 3262434 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0093 GB value = 0.0541 GB dict = 0.54 MB all = 0.0671 GB } __ZipSize{ index = 0.0009 GB value = 0.0178 GB dict = 0.54 MB all = 0.0207 GB } UnZip/Zip{ index = 10.5037 value = 3.0404 dict = 1.00 all = 3.2374 } Zip/UnZip{ index = 0.0952 value = 0.3289 dict = 1.00 all = 0.3089 } ---------------------------- total value len = 0.573150 GB avg = 0.123 KB (by entry num) total key len = 0.130586 GB avg = 0.028 KB total ukey len = 0.093275 GB avg = 0.020 KB total ukey num = 0.004663772 Billion total entry num = 0.004663772 Billion write speed all = 0.070472690 MB/sec (with seq num) write speed all = 0.066736417 MB/sec (without seq num) 2024/08/20-22:57:26.047795 139857135048256 [compaction_job.cc:1944] [default] [JOB 54020] Generated table #14691515: 466062 keys, 20729366 bytes, temperature: kWarm 2024/08/20-22:57:26.047867 139857135048256 EVENT_LOG_v1 {"time_micros": 1724165846047832, "cf_name": "default", "job": 54020, "event": "table_file_creation", "file_number": 14691515, "file_size": 20729366, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 191105706127, "largest_seqno": 191152443576, "table_properties": {"data_size": 17780288, "index_size": 887424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 13049736, "raw_average_key_size": 28, "raw_value_size": 54059168, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 466062, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1724165844, "oldest_key_time": 0, "file_creation_time": 1724165844, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "90ZT1ZM5QNL093I8HHQG", "orig_file_number": 14691515, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-22:57:26.072454 139857135048256 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691516.sst) /dev/shm/ToplingTemp/Topling-50207-Z4ZH3A 2024/08/20-22:57:26.204196 139857135048256 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691516.sst): first pass time = 0.13's, 254.567'MB/sec 2024/08/20-22:57:26.522521 139857135048256 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691516.sst): old prealloc_size = 73819750, real_size = 9499088 2024/08/20-22:57:26.522739 139857135048256 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691516.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 233757 prefix = 4 raw-key = 0.0037 GB zip-key = 0.0004 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0270 GB zip-val = 0.0090 GB avg-val = 115.42 avg-zval = 38.47 2024/08/20-22:57:26.522980 139857135048256 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691516.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.034 GB 2024/08/20-22:57:26.710471 139857135048256 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00/14691516.sst): second pass time = 0.32's, 84.964'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.19's, 55.761'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2423.865'MB/sec (index lex order gen) rebuild zvType time = 0.15's, 0.382'MB/sec write SST data time = 0.03's, 302.368'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.278'MB, throughput = 6158.655'MB/sec zip my value time = 0.32's, unzip length = 0.027'GB zip my value throughput = 84.964'MB/sec zip pipeline throughput = 84.964'MB/sec entries = 233757 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.42 avg-zval = 38.47 usrkeys = 233757 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.42 avg-zval = 38.47 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 233757 bytes = 759728 } seq expand size = 1636299 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0047 GB value = 0.0270 GB dict = 0.28 MB all = 0.0335 GB } __ZipSize{ index = 0.0004 GB value = 0.0090 GB dict = 0.28 MB all = 0.0105 GB } UnZip/Zip{ index = 10.4162 value = 3.0005 dict = 1.00 all = 3.1988 } Zip/UnZip{ index = 0.0960 value = 0.3333 dict = 1.00 all = 0.3126 } ---------------------------- total value len = 0.601766 GB avg = 0.123 KB (by entry num) total key len = 0.137131 GB avg = 0.028 KB total ukey len = 0.097951 GB avg = 0.020 KB total ukey num = 0.004897529 Billion total entry num = 0.004897529 Billion write speed all = 0.073988625 MB/sec (with seq num) write speed all = 0.070065355 MB/sec (without seq num) 2024/08/20-22:57:26.728033 139857135048256 [compaction_job.cc:1944] [default] [JOB 54020] Generated table #14691516: 233757 keys, 10480394 bytes, temperature: kWarm 2024/08/20-22:57:26.728097 139857135048256 EVENT_LOG_v1 {"time_micros": 1724165846728066, "cf_name": "default", "job": 54020, "event": "table_file_creation", "file_number": 14691516, "file_size": 10480394, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 191105706465, "largest_seqno": 191151302064, "table_properties": {"data_size": 8991616, "index_size": 448832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 6545196, "raw_average_key_size": 28, "raw_value_size": 26979510, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 233757, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1724165846, "oldest_key_time": 0, "file_creation_time": 1724165846, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "90ZT1ZM5QNL093I8HHQG", "orig_file_number": 14691516, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-22:57:26.737236 139857135048256 [compaction_job.cc:699] job-54020: subcompact[0], size: 0.219143 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-54020/att-00 [14691506,14691507,14691508,14691509,14691510,14691511,14691512,14691513,14691514,14691515,14691516] 2024/08/20-22:57:26.788791 139857135048256 (Original Log Time 2024/08/20-22:57:10.742350) EVENT_LOG_v1 {"time_micros": 1724165830742300, "job": 54020, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L1": [14691488], "files_L2": [14691412, 14691413, 14691414, 14691415, 14691448, 14691449, 14691450, 14691451, 14691452, 14691453], "score": 0.991228, "input_data_size": 241718871, "oldest_snapshot_seqno": 191152460934} 2024/08/20-22:57:26.788795 139857135048256 (Original Log Time 2024/08/20-22:57:26.787821) [compaction_job.cc:2018] [default] [JOB 54020] Compacted 1@1 + 10@2 files to L2 => 219142886 bytes 2024/08/20-22:57:26.788797 139857135048256 (Original Log Time 2024/08/20-22:57:26.788716) [compaction_job.cc:1220] [default] compacted to: files[0 0 11 0 0 0 0] max score 0.08, MB/sec: 15.1 rd, 13.7 wr, level 2, files in(1, 10) out(11 +0 blob) MB in(31.8, 198.7 +0.0 blob) out(209.0 +0.0 blob), read-write-amplify(13.8) write-amplify(6.6) OK, records in: 4897529, records dropped: 0 output_compression: Snappy 2024/08/20-22:57:26.788803 139857135048256 (Original Log Time 2024/08/20-22:57:26.788765) EVENT_LOG_v1 {"time_micros": 1724165846788743, "cf": "default", "job": 54020, "event": "compaction_finished", "compaction_time_micros": 15994895, "compaction_time_cpu_micros": 6594770, "output_level": 2, "num_output_files": 11, "total_output_size": 219142886, "num_input_records": 4897529, "num_output_records": 4897529, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 151882195, "file_range_sync_nanos": 0, "file_fsync_nanos": 8342, "file_prepare_write_nanos": 34309, "lsm_state": [0, 0, 11, 0, 0, 0, 0]} 2024/08/20-22:57:26.909132 139857135048256 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"a007a635-5c0a-4b4a-be96-b9c695cb016b","dbStarts":1724092920,"starts":1724165830,"executesMs":16172,"compactionJobId":54020,"attempt":0,"compactionInputRawBytes":704614018,"compactionInputZipBytes":241718871,"compactionOutputRawBytes":0,"compactionOutputZipBytes":219142886,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-ac77372cb66f69c70277150da925507f-15efee27d40f7cf8-00"}}, response = 2024/08/20-22:57:26.909230 139857135048256 INFO dcompact_worker.cpp:1456: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-08-20T02.42.00/.rocksdb/job-54020/att-00: olev 2, work 16.173 s, result 5.200 ms, install 1.011 ms, input{raw 671.972 MiB zip 230.521 MiB}