2024/08/20-00:47:25.046601 140500515026496 DEBG dcompact_worker.cpp:2211: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00: fork to child time = 0.010431 sec 2024/08/20-00:47:25.047704 140500515026496 DEBG dcompact_worker.cpp:1011: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00 2024/08/20-00:47:25.055082 140500515026496 DEBG dcompact_worker.cpp:1031: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00 2024/08/20-00:47:25.056971 140500515026496 TRAC dcompact_worker.cpp:1081: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/08/20-00:47:25.057300 140500515026496 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/08/20-00:47:25.096456 140500515026496 [version_set.cc:6139] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-08-19T23.41.30/.rocksdb/job-03104/att-00/MANIFEST-14619535 2024/08/20-00:47:25.151375 140500515026496 [version_set.cc:6182] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-08-19T23.41.30/.rocksdb/job-03104/att-00/MANIFEST-14619535 succeeded,manifest_file_number is 14619535, next_file_number is 14623074, last_sequence is 189658940141, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 14623053 2024/08/20-00:47:25.151390 140500515026496 [version_set.cc:6197] Column family [default] (ID 0), log number is 0 2024/08/20-00:47:25.151567 140500515026496 [version_set.cc:5652] Creating manifest 14623074 2024/08/20-00:47:25.177350 140500515026496 DEBG dcompact_worker.cpp:1215: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00: bottommost_level: fake = 1, rpc = 0 2024/08/20-00:47:25.193904 140500515026496 [compaction_job.cc:2433] [default] [JOB 3104] Compacting 28@4 files to L5, score -1.00, subcompactions 1 : 1 2024/08/20-00:47:25.193951 140500515026496 [compaction_job.cc:2441] [default]: Compaction start summary: Base version 1556 Base level 4, inputs: [14622577(34MB) 14622578(172MB) 14622579(172MB) 14622580(172MB) 14622581(172MB) 14622582(172MB) 14622583(172MB) 14622584(34MB) 14622585(172MB) 14622586(172MB) 14622587(172MB) 14622588(172MB) 14622589(172MB) 14622590(172MB) 14622591(30MB) 14622592(172MB) 14622593(172MB) 14622594(172MB) 14622595(172MB) 14622596(172MB) 14622671(172MB) 14622672(25MB) 14622673(172MB) 14622674(172MB) 14622675(172MB) 14622676(172MB) 14622677(172MB) 14622678(172MB)] 2024/08/20-00:47:25.198461 140500515026496 [top_zip_table.cc:59] core git_version_hash_info_is:commit 7517bad5c1a147474c3830831b9a22c996321ae8 Author: leipeng Date: 2024-01-08 19:18:17 +0800 vfork_cmd: call close before wait_finish g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/08/20-00:47:25.198505 140500515026496 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit 7517bad5c1a147474c3830831b9a22c996321ae8 Author: leipeng Date: 2024-01-08 19:18:17 +0800 vfork_cmd: call close before wait_finish g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/08/20-00:47:25.198531 140500515026496 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit 7517bad5c1a147474c3830831b9a22c996321ae8 Author: leipeng Date: 2024-01-08 19:18:17 +0800 vfork_cmd: call close before wait_finish g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/08/20-00:47:25.198546 140500515026496 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit 7f80399bfbd2071b6e0f5683e8ab7225b8e6e394 Author: leipeng Date: 2024-01-19 14:52:30 +0800 Change ToplingZipTableOptions::sampleRatio to vector g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/08/20-00:47:25.198565 140500515026496 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit 1240987ca13d0fe562c9b9c037155079f0877840 Author: leipeng Date: 2024-01-19 15:07:04 +0800 Change ToplingZipTableOptions::sampleRatio to vector g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/08/20-00:47:25.450306 140500515026496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623075.sst) /dev/shm/ToplingTemp/Topling-82985-ePIPua 2024/08/20-00:47:26.748712 140500515026496 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623075.sst): first pass time = 1.30's, 29.769'MB/sec 2024/08/20-00:47:27.052163 140500515026496 [top_zip_table_builder.cc:2718] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623075.sst): part-0: reject ZipStore ratio = 0.805 2024/08/20-00:47:27.135703 140500515026496 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623075.sst): old prealloc_size = 590558003, real_size = 35681000 2024/08/20-00:47:27.135764 140500515026496 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623075.sst): index type = UintIndex_IL_256_32 , store type = MixedLenBlobStore usrkeys = 193255 prefix = 4 raw-key = 0.0008 GB zip-key = 0.0001 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.0356 GB zip-val = 0.0356 GB avg-val = 184.00 avg-zval = 184.00 2024/08/20-00:47:27.199429 140500515026496 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623075.sst): second pass time = 0.30's, 117.536'MB/sec, value only(92.0% of KV) wait indexing time = 0.08's, remap KeyValue time = 0.06's, 569.764'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1655.981'MB/sec (index lex order gen) rebuild zvType time = 0.04's, 1.279'MB/sec write SST data time = 0.03's, 1401.995'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 0.30's, unzip length = 0.036'GB zip my value throughput = 117.536'MB/sec zip pipeline throughput = 117.536'MB/sec entries = 193255 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 184.00 usrkeys = 193255 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 184.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 193255 bytes = 676400 } seq expand size = 1352785 multi value expand size = 0 cnt WriteAppend = 4348 UnZipSize{ index = 0.0015 GB value = 0.0356 GB dict = 0.00 MB all = 0.0387 GB } __ZipSize{ index = 0.0001 GB value = 0.0356 GB dict = 0.00 MB all = 0.0363 GB } UnZip/Zip{ index = 21.0609 value = 1.0000 dict = 0.00 all = 1.0645 } Zip/UnZip{ index = 0.0475 value = 1.0000 dict = 0.00 all = 0.9394 } ---------------------------- total value len = 62.333661 GB avg = 0.180 KB (by entry num) total key len = 6.791850 GB avg = 0.020 KB total ukey len = 3.191054 GB avg = 0.008 KB total ukey num = 0.389526816 Billion total entry num = 0.346404824 Billion write speed all = 2.658060421 MB/sec (with seq num) write speed all = 2.551498900 MB/sec (without seq num) 2024/08/20-00:47:27.244719 140500515026496 [compaction_job.cc:1944] [default] [JOB 3104] Generated table #14623075: 193255 keys, 36310122 bytes, temperature: kWarm 2024/08/20-00:47:27.244806 140500515026496 EVENT_LOG_v1 {"time_micros": 1724086047244757, "cf_name": "default", "job": 3104, "event": "table_file_creation", "file_number": 14623075, "file_size": 36310122, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189412282135, "largest_seqno": 189634998182, "table_properties": {"data_size": 35559120, "index_size": 73408, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3092080, "raw_average_key_size": 16, "raw_value_size": 35558920, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 193255, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-MixedLenBlobStore", "creation_time": 1724086045, "oldest_key_time": 0, "file_creation_time": 1724086045, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14623075, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:47:27.269850 140500515026496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623076.sst) /dev/shm/ToplingTemp/Topling-82985-V39wFa 2024/08/20-00:47:30.762648 140500515026496 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623076.sst): first pass time = 3.49's, 153.710'MB/sec 2024/08/20-00:47:48.255341 140500515026496 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623076.sst): old prealloc_size = 590558003, real_size = 331404928 2024/08/20-00:47:48.255436 140500515026496 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623076.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2684355 prefix = 4 raw-key = 0.0107 GB zip-key = 0.0010 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.4939 GB zip-val = 0.3297 GB avg-val = 184.00 avg-zval = 122.83 2024/08/20-00:47:50.437439 140500515026496 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623076.sst): second pass time = 17.49's, 28.241'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.18's, 158.126'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 29585.134'MB/sec (index lex order gen) rebuild zvType time = 1.81's, 0.371'MB/sec write SST data time = 0.38's, 918.966'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.914'MB, throughput = 86825.762'MB/sec zip my value time = 17.49's, unzip length = 0.494'GB zip my value throughput = 28.241'MB/sec zip pipeline throughput = 150.227'MB/sec entries = 2684355 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 122.83 usrkeys = 2684355 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 122.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2684355 bytes = 9395264 } seq expand size = 18790485 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0215 GB value = 0.4939 GB dict = 4.91 MB all = 0.5369 GB } __ZipSize{ index = 0.0010 GB value = 0.3297 GB dict = 4.91 MB all = 0.3450 GB } UnZip/Zip{ index = 21.1087 value = 1.4980 dict = 1.00 all = 1.5559 } Zip/UnZip{ index = 0.0474 value = 0.6675 dict = 1.00 all = 0.6427 } ---------------------------- total value len = 62.846373 GB avg = 0.180 KB (by entry num) total key len = 6.834799 GB avg = 0.020 KB total ukey len = 3.212528 GB avg = 0.008 KB total ukey num = 0.392211171 Billion total entry num = 0.349089179 Billion write speed all = 2.677034987 MB/sec (with seq num) write speed all = 2.569743573 MB/sec (without seq num) 2024/08/20-00:47:50.819482 140500515026496 [compaction_job.cc:1944] [default] [JOB 3104] Generated table #14623076: 2684355 keys, 345044107 bytes, temperature: kWarm 2024/08/20-00:47:50.819552 140500515026496 EVENT_LOG_v1 {"time_micros": 1724086070819518, "cf_name": "default", "job": 3104, "event": "table_file_creation", "file_number": 14623076, "file_size": 345044107, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189412281819, "largest_seqno": 189634997855, "table_properties": {"data_size": 329716368, "index_size": 1017344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 42949680, "raw_average_key_size": 16, "raw_value_size": 493921320, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 2684355, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724086047, "oldest_key_time": 0, "file_creation_time": 1724086047, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14623076, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:47:50.934543 140500515026496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623077.sst) /dev/shm/ToplingTemp/Topling-82985-jAx4Y7 2024/08/20-00:47:55.842095 140500515026496 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623077.sst): first pass time = 4.91's, 109.398'MB/sec 2024/08/20-00:48:01.369234 140500515026496 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623077.sst): old prealloc_size = 590558003, real_size = 331397104 2024/08/20-00:48:01.369317 140500515026496 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623077.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2684355 prefix = 4 raw-key = 0.0107 GB zip-key = 0.0010 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.4939 GB zip-val = 0.3297 GB avg-val = 184.00 avg-zval = 122.83 2024/08/20-00:48:03.542093 140500515026496 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623077.sst): second pass time = 5.52's, 89.405'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.17's, 158.794'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 31103.828'MB/sec (index lex order gen) rebuild zvType time = 1.80's, 0.373'MB/sec write SST data time = 0.37's, 922.091'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.914'MB, throughput = 104968.791'MB/sec zip my value time = 5.52's, unzip length = 0.494'GB zip my value throughput = 89.405'MB/sec zip pipeline throughput = 89.405'MB/sec entries = 2684355 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 122.83 usrkeys = 2684355 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 122.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2684355 bytes = 9395264 } seq expand size = 18790485 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0215 GB value = 0.4939 GB dict = 4.91 MB all = 0.5369 GB } __ZipSize{ index = 0.0010 GB value = 0.3297 GB dict = 4.91 MB all = 0.3450 GB } UnZip/Zip{ index = 21.1087 value = 1.4981 dict = 1.00 all = 1.5560 } Zip/UnZip{ index = 0.0474 value = 0.6675 dict = 1.00 all = 0.6427 } ---------------------------- total value len = 63.359085 GB avg = 0.180 KB (by entry num) total key len = 6.877749 GB avg = 0.020 KB total ukey len = 3.234003 GB avg = 0.008 KB total ukey num = 0.394895526 Billion total entry num = 0.351773534 Billion write speed all = 2.697024738 MB/sec (with seq num) write speed all = 2.588962701 MB/sec (without seq num) 2024/08/20-00:48:03.912172 140500515026496 [compaction_job.cc:1944] [default] [JOB 3104] Generated table #14623077: 2684355 keys, 345036299 bytes, temperature: kWarm 2024/08/20-00:48:03.912244 140500515026496 EVENT_LOG_v1 {"time_micros": 1724086083912209, "cf_name": "default", "job": 3104, "event": "table_file_creation", "file_number": 14623077, "file_size": 345036299, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189412281821, "largest_seqno": 189634997854, "table_properties": {"data_size": 329708544, "index_size": 1017344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 42949680, "raw_average_key_size": 16, "raw_value_size": 493921320, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 2684355, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724086070, "oldest_key_time": 0, "file_creation_time": 1724086070, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14623077, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:48:04.021856 140500515026496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623078.sst) /dev/shm/ToplingTemp/Topling-82985-8GyuX8 2024/08/20-00:48:08.709202 140500515026496 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623078.sst): first pass time = 4.69's, 114.537'MB/sec 2024/08/20-00:48:14.112694 140500515026496 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623078.sst): old prealloc_size = 590558003, real_size = 331417088 2024/08/20-00:48:14.112778 140500515026496 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623078.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2684355 prefix = 4 raw-key = 0.0107 GB zip-key = 0.0010 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.4939 GB zip-val = 0.3297 GB avg-val = 184.00 avg-zval = 122.83 2024/08/20-00:48:16.306514 140500515026496 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623078.sst): second pass time = 5.40's, 91.452'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.19's, 157.286'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 31722.952'MB/sec (index lex order gen) rebuild zvType time = 1.82's, 0.369'MB/sec write SST data time = 0.38's, 914.889'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.914'MB, throughput = 86649.691'MB/sec zip my value time = 5.40's, unzip length = 0.494'GB zip my value throughput = 91.452'MB/sec zip pipeline throughput = 91.452'MB/sec entries = 2684355 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 122.83 usrkeys = 2684355 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 122.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2684355 bytes = 9395264 } seq expand size = 18790485 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0215 GB value = 0.4939 GB dict = 4.91 MB all = 0.5369 GB } __ZipSize{ index = 0.0010 GB value = 0.3297 GB dict = 4.91 MB all = 0.3451 GB } UnZip/Zip{ index = 21.0822 value = 1.4980 dict = 1.00 all = 1.5559 } Zip/UnZip{ index = 0.0474 value = 0.6676 dict = 1.00 all = 0.6427 } ---------------------------- total value len = 63.871797 GB avg = 0.180 KB (by entry num) total key len = 6.920699 GB avg = 0.020 KB total ukey len = 3.255478 GB avg = 0.008 KB total ukey num = 0.397579881 Billion total entry num = 0.354457889 Billion write speed all = 2.717029859 MB/sec (with seq num) write speed all = 2.608196553 MB/sec (without seq num) 2024/08/20-00:48:16.678882 140500515026496 [compaction_job.cc:1944] [default] [JOB 3104] Generated table #14623078: 2684355 keys, 345056267 bytes, temperature: kWarm 2024/08/20-00:48:16.678979 140500515026496 EVENT_LOG_v1 {"time_micros": 1724086096678936, "cf_name": "default", "job": 3104, "event": "table_file_creation", "file_number": 14623078, "file_size": 345056267, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189412281825, "largest_seqno": 189634997853, "table_properties": {"data_size": 329727248, "index_size": 1018624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 42949680, "raw_average_key_size": 16, "raw_value_size": 493921320, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 2684355, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724086084, "oldest_key_time": 0, "file_creation_time": 1724086084, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14623078, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:48:16.791599 140500515026496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623079.sst) /dev/shm/ToplingTemp/Topling-82985-c3zRu6 2024/08/20-00:48:18.369630 140500515026496 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623079.sst): first pass time = 1.58's, 24.325'MB/sec 2024/08/20-00:48:18.665276 140500515026496 [top_zip_table_builder.cc:2718] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623079.sst): part-0: reject ZipStore ratio = 0.806 2024/08/20-00:48:18.745694 140500515026496 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623079.sst): old prealloc_size = 590558003, real_size = 35435336 2024/08/20-00:48:18.745744 140500515026496 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623079.sst): index type = UintIndex_IL_256_32 , store type = MixedLenBlobStore usrkeys = 191925 prefix = 4 raw-key = 0.0008 GB zip-key = 0.0001 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.0353 GB zip-val = 0.0353 GB avg-val = 184.00 avg-zval = 184.00 2024/08/20-00:48:18.807057 140500515026496 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623079.sst): second pass time = 0.29's, 119.772'MB/sec, value only(92.0% of KV) wait indexing time = 0.08's, remap KeyValue time = 0.06's, 587.678'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2216.501'MB/sec (index lex order gen) rebuild zvType time = 0.04's, 1.353'MB/sec write SST data time = 0.03's, 1393.543'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 0.29's, unzip length = 0.035'GB zip my value throughput = 119.772'MB/sec zip pipeline throughput = 119.772'MB/sec entries = 191925 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 184.00 usrkeys = 191925 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 184.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 191925 bytes = 671760 } seq expand size = 1343475 multi value expand size = 0 cnt WriteAppend = 4318 UnZipSize{ index = 0.0015 GB value = 0.0353 GB dict = 0.00 MB all = 0.0384 GB } __ZipSize{ index = 0.0001 GB value = 0.0353 GB dict = 0.00 MB all = 0.0361 GB } UnZip/Zip{ index = 21.0814 value = 1.0000 dict = 0.00 all = 1.0645 } Zip/UnZip{ index = 0.0474 value = 1.0000 dict = 0.00 all = 0.9394 } ---------------------------- total value len = 63.908455 GB avg = 0.180 KB (by entry num) total key len = 6.923770 GB avg = 0.020 KB total ukey len = 3.257013 GB avg = 0.008 KB total ukey num = 0.397771806 Billion total entry num = 0.354649814 Billion write speed all = 2.718293768 MB/sec (with seq num) write speed all = 2.609411982 MB/sec (without seq num) 2024/08/20-00:48:18.850202 140500515026496 [compaction_job.cc:1944] [default] [JOB 3104] Generated table #14623079: 191925 keys, 36060202 bytes, temperature: kWarm 2024/08/20-00:48:18.850272 140500515026496 EVENT_LOG_v1 {"time_micros": 1724086098850238, "cf_name": "default", "job": 3104, "event": "table_file_creation", "file_number": 14623079, "file_size": 36060202, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189412281823, "largest_seqno": 189634997524, "table_properties": {"data_size": 35314400, "index_size": 72832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3070800, "raw_average_key_size": 16, "raw_value_size": 35314200, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 191925, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-MixedLenBlobStore", "creation_time": 1724086096, "oldest_key_time": 0, "file_creation_time": 1724086096, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14623079, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:48:18.875110 140500515026496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623080.sst) /dev/shm/ToplingTemp/Topling-82985-hpMTha 2024/08/20-00:48:22.379533 140500515026496 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623080.sst): first pass time = 3.50's, 153.200'MB/sec 2024/08/20-00:48:27.758524 140500515026496 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623080.sst): old prealloc_size = 590558003, real_size = 331388408 2024/08/20-00:48:27.758613 140500515026496 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623080.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2684355 prefix = 4 raw-key = 0.0107 GB zip-key = 0.0010 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.4939 GB zip-val = 0.3297 GB avg-val = 184.00 avg-zval = 122.83 2024/08/20-00:48:29.946530 140500515026496 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623080.sst): second pass time = 5.38's, 91.870'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.19's, 157.691'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 28734.137'MB/sec (index lex order gen) rebuild zvType time = 1.82's, 0.370'MB/sec write SST data time = 0.37's, 925.562'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.914'MB, throughput = 87765.525'MB/sec zip my value time = 5.38's, unzip length = 0.494'GB zip my value throughput = 91.870'MB/sec zip pipeline throughput = 91.870'MB/sec entries = 2684355 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 122.83 usrkeys = 2684355 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 122.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2684355 bytes = 9395264 } seq expand size = 18790485 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0215 GB value = 0.4939 GB dict = 4.91 MB all = 0.5369 GB } __ZipSize{ index = 0.0010 GB value = 0.3297 GB dict = 4.91 MB all = 0.3450 GB } UnZip/Zip{ index = 21.2464 value = 1.4981 dict = 1.00 all = 1.5560 } Zip/UnZip{ index = 0.0471 value = 0.6675 dict = 1.00 all = 0.6427 } ---------------------------- total value len = 64.421166 GB avg = 0.180 KB (by entry num) total key len = 6.966719 GB avg = 0.019 KB total ukey len = 3.278488 GB avg = 0.008 KB total ukey num = 0.400456161 Billion total entry num = 0.357334169 Billion write speed all = 2.738447448 MB/sec (with seq num) write speed all = 2.628788412 MB/sec (without seq num) 2024/08/20-00:48:30.322207 140500515026496 [compaction_job.cc:1944] [default] [JOB 3104] Generated table #14623080: 2684355 keys, 345027595 bytes, temperature: kWarm 2024/08/20-00:48:30.322280 140500515026496 EVENT_LOG_v1 {"time_micros": 1724086110322242, "cf_name": "default", "job": 3104, "event": "table_file_creation", "file_number": 14623080, "file_size": 345027595, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189410454032, "largest_seqno": 189634998227, "table_properties": {"data_size": 329706448, "index_size": 1010752, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 42949680, "raw_average_key_size": 16, "raw_value_size": 493921320, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 2684355, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724086098, "oldest_key_time": 0, "file_creation_time": 1724086098, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14623080, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:48:30.433370 140500515026496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623081.sst) /dev/shm/ToplingTemp/Topling-82985-ZfklG8 2024/08/20-00:48:35.104737 140500515026496 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623081.sst): first pass time = 4.67's, 114.929'MB/sec 2024/08/20-00:48:40.424275 140500515026496 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623081.sst): old prealloc_size = 590558003, real_size = 331406504 2024/08/20-00:48:40.424361 140500515026496 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623081.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2684355 prefix = 4 raw-key = 0.0107 GB zip-key = 0.0010 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.4939 GB zip-val = 0.3297 GB avg-val = 184.00 avg-zval = 122.83 2024/08/20-00:48:42.601248 140500515026496 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623081.sst): second pass time = 5.32's, 92.893'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.18's, 158.500'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 28514.283'MB/sec (index lex order gen) rebuild zvType time = 1.80's, 0.372'MB/sec write SST data time = 0.37's, 924.228'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.914'MB, throughput = 105065.298'MB/sec zip my value time = 5.32's, unzip length = 0.494'GB zip my value throughput = 92.893'MB/sec zip pipeline throughput = 92.893'MB/sec entries = 2684355 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 122.83 usrkeys = 2684355 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 122.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2684355 bytes = 9395264 } seq expand size = 18790485 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0215 GB value = 0.4939 GB dict = 4.91 MB all = 0.5369 GB } __ZipSize{ index = 0.0010 GB value = 0.3297 GB dict = 4.91 MB all = 0.3450 GB } UnZip/Zip{ index = 21.1327 value = 1.4980 dict = 1.00 all = 1.5559 } Zip/UnZip{ index = 0.0473 value = 0.6676 dict = 1.00 all = 0.6427 } ---------------------------- total value len = 64.933878 GB avg = 0.180 KB (by entry num) total key len = 7.009669 GB avg = 0.019 KB total ukey len = 3.299963 GB avg = 0.008 KB total ukey num = 0.403140516 Billion total entry num = 0.360018524 Billion write speed all = 2.758423648 MB/sec (with seq num) write speed all = 2.647994441 MB/sec (without seq num) 2024/08/20-00:48:42.975783 140500515026496 [compaction_job.cc:1944] [default] [JOB 3104] Generated table #14623081: 2684355 keys, 345045707 bytes, temperature: kWarm 2024/08/20-00:48:42.975863 140500515026496 EVENT_LOG_v1 {"time_micros": 1724086122975818, "cf_name": "default", "job": 3104, "event": "table_file_creation", "file_number": 14623081, "file_size": 345045707, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189410454457, "largest_seqno": 189634998231, "table_properties": {"data_size": 329719104, "index_size": 1016192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 42949680, "raw_average_key_size": 16, "raw_value_size": 493921320, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 2684355, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724086110, "oldest_key_time": 0, "file_creation_time": 1724086110, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14623081, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:48:43.084680 140500515026496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623082.sst) /dev/shm/ToplingTemp/Topling-82985-LW7uv6 2024/08/20-00:48:47.750611 140500515026496 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623082.sst): first pass time = 4.67's, 115.063'MB/sec 2024/08/20-00:48:53.071633 140500515026496 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623082.sst): old prealloc_size = 590558003, real_size = 331394728 2024/08/20-00:48:53.071729 140500515026496 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623082.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2684355 prefix = 4 raw-key = 0.0107 GB zip-key = 0.0010 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.4939 GB zip-val = 0.3297 GB avg-val = 184.00 avg-zval = 122.82 2024/08/20-00:48:55.850982 140500515026496 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623082.sst): second pass time = 5.32's, 92.871'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.78's, 124.143'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 30085.066'MB/sec (index lex order gen) rebuild zvType time = 2.28's, 0.295'MB/sec write SST data time = 0.50's, 685.656'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.914'MB, throughput = 83963.912'MB/sec zip my value time = 5.32's, unzip length = 0.494'GB zip my value throughput = 92.871'MB/sec zip pipeline throughput = 92.871'MB/sec entries = 2684355 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 122.82 usrkeys = 2684355 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 122.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2684355 bytes = 9395264 } seq expand size = 18790485 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0215 GB value = 0.4939 GB dict = 4.91 MB all = 0.5369 GB } __ZipSize{ index = 0.0010 GB value = 0.3297 GB dict = 4.91 MB all = 0.3450 GB } UnZip/Zip{ index = 21.0835 value = 1.4981 dict = 1.00 all = 1.5560 } Zip/UnZip{ index = 0.0474 value = 0.6675 dict = 1.00 all = 0.6427 } ---------------------------- total value len = 65.446590 GB avg = 0.180 KB (by entry num) total key len = 7.052619 GB avg = 0.019 KB total ukey len = 3.321438 GB avg = 0.008 KB total ukey num = 0.405824871 Billion total entry num = 0.362702879 Billion write speed all = 2.778317118 MB/sec (with seq num) write speed all = 2.667121022 MB/sec (without seq num) 2024/08/20-00:48:56.411209 140500515026496 [compaction_job.cc:1944] [default] [JOB 3104] Generated table #14623082: 2684355 keys, 345033931 bytes, temperature: kWarm 2024/08/20-00:48:56.411280 140500515026496 EVENT_LOG_v1 {"time_micros": 1724086136411245, "cf_name": "default", "job": 3104, "event": "table_file_creation", "file_number": 14623082, "file_size": 345033931, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189412281725, "largest_seqno": 189634998230, "table_properties": {"data_size": 329704928, "index_size": 1018560, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 42949680, "raw_average_key_size": 16, "raw_value_size": 493921320, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 2684355, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724086123, "oldest_key_time": 0, "file_creation_time": 1724086123, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14623082, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:48:56.524465 140500515026496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623083.sst) /dev/shm/ToplingTemp/Topling-82985-9Licc8 2024/08/20-00:48:59.627549 140500515026496 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623083.sst): first pass time = 3.10's, 13.663'MB/sec 2024/08/20-00:49:02.690798 140500515026496 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623083.sst): old prealloc_size = 590558003, real_size = 31305744 2024/08/20-00:49:02.690895 140500515026496 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623083.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 211981 prefix = 4 raw-key = 0.0008 GB zip-key = 0.0001 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.0390 GB zip-val = 0.0312 GB avg-val = 184.00 avg-zval = 147.05 2024/08/20-00:49:02.996706 140500515026496 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623083.sst): second pass time = 3.06's, 12.737'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.31's, 105.920'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2081.408'MB/sec (index lex order gen) rebuild zvType time = 0.25's, 0.214'MB/sec write SST data time = 0.06's, 561.946'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.405'MB, throughput = 8777.667'MB/sec zip my value time = 3.06's, unzip length = 0.039'GB zip my value throughput = 12.737'MB/sec zip pipeline throughput = 158.659'MB/sec entries = 211981 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 147.05 usrkeys = 211981 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 147.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 211981 bytes = 741952 } seq expand size = 1483867 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0017 GB value = 0.0390 GB dict = 0.40 MB all = 0.0424 GB } __ZipSize{ index = 0.0001 GB value = 0.0312 GB dict = 0.40 MB all = 0.0324 GB } UnZip/Zip{ index = 21.0299 value = 1.2513 dict = 1.00 all = 1.3085 } Zip/UnZip{ index = 0.0476 value = 0.7992 dict = 1.00 all = 0.7642 } ---------------------------- total value len = 65.487078 GB avg = 0.180 KB (by entry num) total key len = 7.056010 GB avg = 0.019 KB total ukey len = 3.323134 GB avg = 0.008 KB total ukey num = 0.406036852 Billion total entry num = 0.362914860 Billion write speed all = 2.779237628 MB/sec (with seq num) write speed all = 2.668007003 MB/sec (without seq num) 2024/08/20-00:49:03.039343 140500515026496 [compaction_job.cc:1944] [default] [JOB 3104] Generated table #14623083: 211981 keys, 32400779 bytes, temperature: kWarm 2024/08/20-00:49:03.039425 140500515026496 EVENT_LOG_v1 {"time_micros": 1724086143039382, "cf_name": "default", "job": 3104, "event": "table_file_creation", "file_number": 14623083, "file_size": 32400779, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189412281719, "largest_seqno": 189634995626, "table_properties": {"data_size": 31171952, "index_size": 80640, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3391696, "raw_average_key_size": 16, "raw_value_size": 39004504, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 211981, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724086136, "oldest_key_time": 0, "file_creation_time": 1724086136, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14623083, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:49:03.068271 140500515026496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623084.sst) /dev/shm/ToplingTemp/Topling-82985-mbOAO9 2024/08/20-00:49:09.093616 140500515026496 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623084.sst): first pass time = 6.03's, 89.103'MB/sec 2024/08/20-00:49:15.617625 140500515026496 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623084.sst): old prealloc_size = 590558003, real_size = 331384720 2024/08/20-00:49:15.617710 140500515026496 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623084.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2684355 prefix = 4 raw-key = 0.0107 GB zip-key = 0.0010 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.4939 GB zip-val = 0.3297 GB avg-val = 184.00 avg-zval = 122.82 2024/08/20-00:49:17.817514 140500515026496 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623084.sst): second pass time = 6.52's, 75.743'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.20's, 156.837'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 28999.433'MB/sec (index lex order gen) rebuild zvType time = 1.82's, 0.369'MB/sec write SST data time = 0.38's, 900.476'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.914'MB, throughput = 91542.391'MB/sec zip my value time = 6.52's, unzip length = 0.494'GB zip my value throughput = 75.743'MB/sec zip pipeline throughput = 104.619'MB/sec entries = 2684355 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 122.82 usrkeys = 2684355 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 122.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2684355 bytes = 9395264 } seq expand size = 18790485 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0215 GB value = 0.4939 GB dict = 4.91 MB all = 0.5369 GB } __ZipSize{ index = 0.0010 GB value = 0.3297 GB dict = 4.91 MB all = 0.3450 GB } UnZip/Zip{ index = 20.9781 value = 1.4981 dict = 1.00 all = 1.5560 } Zip/UnZip{ index = 0.0477 value = 0.6675 dict = 1.00 all = 0.6427 } ---------------------------- total value len = 65.999790 GB avg = 0.181 KB (by entry num) total key len = 7.098960 GB avg = 0.019 KB total ukey len = 3.344609 GB avg = 0.008 KB total ukey num = 0.408721207 Billion total entry num = 0.365599215 Billion write speed all = 2.798936617 MB/sec (with seq num) write speed all = 2.686946846 MB/sec (without seq num) 2024/08/20-00:49:18.196337 140500515026496 [compaction_job.cc:1944] [default] [JOB 3104] Generated table #14623084: 2684355 keys, 345023947 bytes, temperature: kWarm 2024/08/20-00:49:18.196405 140500515026496 EVENT_LOG_v1 {"time_micros": 1724086158196373, "cf_name": "default", "job": 3104, "event": "table_file_creation", "file_number": 14623084, "file_size": 345023947, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189414096783, "largest_seqno": 189634998194, "table_properties": {"data_size": 329689840, "index_size": 1023680, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 42949680, "raw_average_key_size": 16, "raw_value_size": 493921320, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 2684355, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724086143, "oldest_key_time": 0, "file_creation_time": 1724086143, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14623084, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:49:18.307604 140500515026496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623085.sst) /dev/shm/ToplingTemp/Topling-82985-xQRDD8 2024/08/20-00:49:23.144247 140500515026496 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623085.sst): first pass time = 4.84's, 111.002'MB/sec 2024/08/20-00:49:28.586403 140500515026496 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623085.sst): old prealloc_size = 590558003, real_size = 331419896 2024/08/20-00:49:28.586494 140500515026496 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623085.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2684355 prefix = 4 raw-key = 0.0107 GB zip-key = 0.0010 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.4939 GB zip-val = 0.3297 GB avg-val = 184.00 avg-zval = 122.83 2024/08/20-00:49:30.775771 140500515026496 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623085.sst): second pass time = 5.44's, 90.811'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.19's, 157.607'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 30192.026'MB/sec (index lex order gen) rebuild zvType time = 1.81's, 0.371'MB/sec write SST data time = 0.38's, 908.330'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.914'MB, throughput = 114840.356'MB/sec zip my value time = 5.44's, unzip length = 0.494'GB zip my value throughput = 90.811'MB/sec zip pipeline throughput = 90.811'MB/sec entries = 2684355 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 122.83 usrkeys = 2684355 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 122.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2684355 bytes = 9395264 } seq expand size = 18790485 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0215 GB value = 0.4939 GB dict = 4.91 MB all = 0.5369 GB } __ZipSize{ index = 0.0010 GB value = 0.3297 GB dict = 4.91 MB all = 0.3451 GB } UnZip/Zip{ index = 20.9741 value = 1.4980 dict = 1.00 all = 1.5559 } Zip/UnZip{ index = 0.0477 value = 0.6676 dict = 1.00 all = 0.6427 } ---------------------------- total value len = 66.512502 GB avg = 0.181 KB (by entry num) total key len = 7.141910 GB avg = 0.019 KB total ukey len = 3.366083 GB avg = 0.008 KB total ukey num = 0.411405562 Billion total entry num = 0.368283570 Billion write speed all = 2.818814177 MB/sec (with seq num) write speed all = 2.706058085 MB/sec (without seq num) 2024/08/20-00:49:31.151601 140500515026496 [compaction_job.cc:1944] [default] [JOB 3104] Generated table #14623085: 2684355 keys, 345059083 bytes, temperature: kWarm 2024/08/20-00:49:31.151673 140500515026496 EVENT_LOG_v1 {"time_micros": 1724086171151638, "cf_name": "default", "job": 3104, "event": "table_file_creation", "file_number": 14623085, "file_size": 345059083, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189414096779, "largest_seqno": 189634998195, "table_properties": {"data_size": 329724816, "index_size": 1023872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 42949680, "raw_average_key_size": 16, "raw_value_size": 493921320, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 2684355, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724086158, "oldest_key_time": 0, "file_creation_time": 1724086158, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14623085, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:49:31.262757 140500515026496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623086.sst) /dev/shm/ToplingTemp/Topling-82985-Kha3K6 2024/08/20-00:49:35.917882 140500515026496 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623086.sst): first pass time = 4.66's, 115.330'MB/sec 2024/08/20-00:49:41.240817 140500515026496 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623086.sst): old prealloc_size = 590558003, real_size = 331409496 2024/08/20-00:49:41.240916 140500515026496 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623086.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2684355 prefix = 4 raw-key = 0.0107 GB zip-key = 0.0010 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.4939 GB zip-val = 0.3297 GB avg-val = 184.00 avg-zval = 122.83 2024/08/20-00:49:43.430140 140500515026496 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623086.sst): second pass time = 5.32's, 92.841'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.19's, 157.606'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 22212.925'MB/sec (index lex order gen) rebuild zvType time = 1.81's, 0.371'MB/sec write SST data time = 0.38's, 905.790'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.914'MB, throughput = 101967.255'MB/sec zip my value time = 5.32's, unzip length = 0.494'GB zip my value throughput = 92.841'MB/sec zip pipeline throughput = 92.841'MB/sec entries = 2684355 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 122.83 usrkeys = 2684355 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 122.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2684355 bytes = 9395264 } seq expand size = 18790485 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0215 GB value = 0.4939 GB dict = 4.91 MB all = 0.5369 GB } __ZipSize{ index = 0.0010 GB value = 0.3297 GB dict = 4.91 MB all = 0.3450 GB } UnZip/Zip{ index = 20.9794 value = 1.4980 dict = 1.00 all = 1.5559 } Zip/UnZip{ index = 0.0477 value = 0.6675 dict = 1.00 all = 0.6427 } ---------------------------- total value len = 67.025214 GB avg = 0.181 KB (by entry num) total key len = 7.184859 GB avg = 0.019 KB total ukey len = 3.387558 GB avg = 0.008 KB total ukey num = 0.414089917 Billion total entry num = 0.370967925 Billion write speed all = 2.838705028 MB/sec (with seq num) write speed all = 2.725182055 MB/sec (without seq num) 2024/08/20-00:49:43.803768 140500515026496 [compaction_job.cc:1944] [default] [JOB 3104] Generated table #14623086: 2684355 keys, 345048715 bytes, temperature: kWarm 2024/08/20-00:49:43.803850 140500515026496 EVENT_LOG_v1 {"time_micros": 1724086183803806, "cf_name": "default", "job": 3104, "event": "table_file_creation", "file_number": 14623086, "file_size": 345048715, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189414096780, "largest_seqno": 189634998191, "table_properties": {"data_size": 329714656, "index_size": 1023616, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 42949680, "raw_average_key_size": 16, "raw_value_size": 493921320, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 2684355, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724086171, "oldest_key_time": 0, "file_creation_time": 1724086171, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14623086, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:49:43.914711 140500515026496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623087.sst) /dev/shm/ToplingTemp/Topling-82985-xvKaU9 2024/08/20-00:49:45.356563 140500515026496 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623087.sst): first pass time = 1.44's, 19.913'MB/sec 2024/08/20-00:49:45.597779 140500515026496 [top_zip_table_builder.cc:2718] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623087.sst): part-0: reject ZipStore ratio = 0.828 2024/08/20-00:49:45.658978 140500515026496 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623087.sst): old prealloc_size = 590558003, real_size = 26504952 2024/08/20-00:49:45.659029 140500515026496 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623087.sst): index type = UintIndex_IL_256_32 , store type = MixedLenBlobStore usrkeys = 143553 prefix = 4 raw-key = 0.0006 GB zip-key = 0.0001 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.0264 GB zip-val = 0.0264 GB avg-val = 184.00 avg-zval = 184.00 2024/08/20-00:49:45.708979 140500515026496 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623087.sst): second pass time = 0.24's, 109.820'MB/sec, value only(92.0% of KV) wait indexing time = 0.06's, remap KeyValue time = 0.05's, 539.402'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1848.430'MB/sec (index lex order gen) rebuild zvType time = 0.03's, 1.162'MB/sec write SST data time = 0.02's, 1412.748'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 0.24's, unzip length = 0.026'GB zip my value throughput = 109.820'MB/sec zip pipeline throughput = 109.820'MB/sec entries = 143553 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 184.00 usrkeys = 143553 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 184.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 143553 bytes = 502448 } seq expand size = 1004871 multi value expand size = 0 cnt WriteAppend = 3231 UnZipSize{ index = 0.0011 GB value = 0.0264 GB dict = 0.00 MB all = 0.0287 GB } __ZipSize{ index = 0.0001 GB value = 0.0264 GB dict = 0.00 MB all = 0.0270 GB } UnZip/Zip{ index = 20.8896 value = 1.0000 dict = 0.00 all = 1.0644 } Zip/UnZip{ index = 0.0479 value = 1.0000 dict = 0.00 all = 0.9395 } ---------------------------- total value len = 67.052632 GB avg = 0.181 KB (by entry num) total key len = 7.187156 GB avg = 0.019 KB total ukey len = 3.388707 GB avg = 0.008 KB total ukey num = 0.414233470 Billion total entry num = 0.371111478 Billion write speed all = 2.839594184 MB/sec (with seq num) write speed all = 2.726037180 MB/sec (without seq num) 2024/08/20-00:49:45.742981 140500515026496 [compaction_job.cc:1944] [default] [JOB 3104] Generated table #14623087: 143553 keys, 26972522 bytes, temperature: kWarm 2024/08/20-00:49:45.743052 140500515026496 EVENT_LOG_v1 {"time_micros": 1724086185743017, "cf_name": "default", "job": 3104, "event": "table_file_creation", "file_number": 14623087, "file_size": 26972522, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189414098389, "largest_seqno": 189634996325, "table_properties": {"data_size": 26413952, "index_size": 54976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2296848, "raw_average_key_size": 16, "raw_value_size": 26413752, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 143553, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-MixedLenBlobStore", "creation_time": 1724086183, "oldest_key_time": 0, "file_creation_time": 1724086183, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14623087, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:49:45.764319 140500515026496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623088.sst) /dev/shm/ToplingTemp/Topling-82985-93V7e6 2024/08/20-00:49:49.235994 140500515026496 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623088.sst): first pass time = 3.47's, 154.645'MB/sec 2024/08/20-00:49:54.652018 140500515026496 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623088.sst): old prealloc_size = 590558003, real_size = 331409288 2024/08/20-00:49:54.652106 140500515026496 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623088.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2684355 prefix = 4 raw-key = 0.0107 GB zip-key = 0.0010 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.4939 GB zip-val = 0.3297 GB avg-val = 184.00 avg-zval = 122.83 2024/08/20-00:49:56.837459 140500515026496 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623088.sst): second pass time = 5.41's, 91.243'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.19's, 157.886'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 29513.780'MB/sec (index lex order gen) rebuild zvType time = 1.81's, 0.371'MB/sec write SST data time = 0.37's, 921.086'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.914'MB, throughput = 116305.420'MB/sec zip my value time = 5.41's, unzip length = 0.494'GB zip my value throughput = 91.243'MB/sec zip pipeline throughput = 91.243'MB/sec entries = 2684355 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 122.83 usrkeys = 2684355 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 122.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2684355 bytes = 9395264 } seq expand size = 18790485 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0215 GB value = 0.4939 GB dict = 4.91 MB all = 0.5369 GB } __ZipSize{ index = 0.0010 GB value = 0.3297 GB dict = 4.91 MB all = 0.3450 GB } UnZip/Zip{ index = 20.8451 value = 1.4981 dict = 1.00 all = 1.5559 } Zip/UnZip{ index = 0.0480 value = 0.6675 dict = 1.00 all = 0.6427 } ---------------------------- total value len = 67.565344 GB avg = 0.181 KB (by entry num) total key len = 7.230106 GB avg = 0.019 KB total ukey len = 3.410182 GB avg = 0.008 KB total ukey num = 0.416917825 Billion total entry num = 0.373795833 Billion write speed all = 2.859630444 MB/sec (with seq num) write speed all = 2.745300715 MB/sec (without seq num) 2024/08/20-00:49:57.216442 140500515026496 [compaction_job.cc:1944] [default] [JOB 3104] Generated table #14623088: 2684355 keys, 345048523 bytes, temperature: kWarm 2024/08/20-00:49:57.216513 140500515026496 EVENT_LOG_v1 {"time_micros": 1724086197216477, "cf_name": "default", "job": 3104, "event": "table_file_creation", "file_number": 14623088, "file_size": 345048523, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189415896976, "largest_seqno": 189634997914, "table_properties": {"data_size": 329707888, "index_size": 1030208, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 42949680, "raw_average_key_size": 16, "raw_value_size": 493921320, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 2684355, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724086185, "oldest_key_time": 0, "file_creation_time": 1724086185, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14623088, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:49:57.327069 140500515026496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623089.sst) /dev/shm/ToplingTemp/Topling-82985-BUiH28 2024/08/20-00:50:02.020815 140500515026496 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623089.sst): first pass time = 4.69's, 114.381'MB/sec 2024/08/20-00:50:07.466567 140500515026496 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623089.sst): old prealloc_size = 590558003, real_size = 331405024 2024/08/20-00:50:07.466653 140500515026496 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623089.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2684355 prefix = 4 raw-key = 0.0107 GB zip-key = 0.0010 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.4939 GB zip-val = 0.3297 GB avg-val = 184.00 avg-zval = 122.82 2024/08/20-00:50:09.649868 140500515026496 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623089.sst): second pass time = 5.44's, 90.747'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.18's, 158.039'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 30339.876'MB/sec (index lex order gen) rebuild zvType time = 1.81's, 0.371'MB/sec write SST data time = 0.37's, 925.635'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.914'MB, throughput = 88519.671'MB/sec zip my value time = 5.44's, unzip length = 0.494'GB zip my value throughput = 90.747'MB/sec zip pipeline throughput = 90.747'MB/sec entries = 2684355 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 122.82 usrkeys = 2684355 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 122.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2684355 bytes = 9395264 } seq expand size = 18790485 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0215 GB value = 0.4939 GB dict = 4.91 MB all = 0.5369 GB } __ZipSize{ index = 0.0010 GB value = 0.3297 GB dict = 4.91 MB all = 0.3450 GB } UnZip/Zip{ index = 20.8283 value = 1.4981 dict = 1.00 all = 1.5559 } Zip/UnZip{ index = 0.0480 value = 0.6675 dict = 1.00 all = 0.6427 } ---------------------------- total value len = 68.078056 GB avg = 0.181 KB (by entry num) total key len = 7.273056 GB avg = 0.019 KB total ukey len = 3.431656 GB avg = 0.008 KB total ukey num = 0.419602180 Billion total entry num = 0.376480188 Billion write speed all = 2.879464358 MB/sec (with seq num) write speed all = 2.764369968 MB/sec (without seq num) 2024/08/20-00:50:10.027800 140500515026496 [compaction_job.cc:1944] [default] [JOB 3104] Generated table #14623089: 2684355 keys, 345044235 bytes, temperature: kWarm 2024/08/20-00:50:10.027882 140500515026496 EVENT_LOG_v1 {"time_micros": 1724086210027846, "cf_name": "default", "job": 3104, "event": "table_file_creation", "file_number": 14623089, "file_size": 345044235, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189415896977, "largest_seqno": 189634997923, "table_properties": {"data_size": 329702784, "index_size": 1031040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 42949680, "raw_average_key_size": 16, "raw_value_size": 493921320, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 2684355, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724086197, "oldest_key_time": 0, "file_creation_time": 1724086197, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14623089, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:50:10.146286 140500515026496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623090.sst) /dev/shm/ToplingTemp/Topling-82985-u9UGh9 2024/08/20-00:50:15.017747 140500515026496 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623090.sst): first pass time = 4.87's, 110.208'MB/sec 2024/08/20-00:50:20.771148 140500515026496 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623090.sst): old prealloc_size = 590558003, real_size = 331423776 2024/08/20-00:50:20.771245 140500515026496 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623090.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2684355 prefix = 4 raw-key = 0.0107 GB zip-key = 0.0010 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.4939 GB zip-val = 0.3297 GB avg-val = 184.00 avg-zval = 122.83 2024/08/20-00:50:23.775953 140500515026496 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623090.sst): second pass time = 5.75's, 85.909'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 3.00's, 114.838'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 24303.090'MB/sec (index lex order gen) rebuild zvType time = 2.50's, 0.268'MB/sec write SST data time = 0.51's, 682.938'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.914'MB, throughput = 94554.523'MB/sec zip my value time = 5.75's, unzip length = 0.494'GB zip my value throughput = 85.909'MB/sec zip pipeline throughput = 85.909'MB/sec entries = 2684355 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 122.83 usrkeys = 2684355 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 122.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2684355 bytes = 9395264 } seq expand size = 18790485 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0215 GB value = 0.4939 GB dict = 4.91 MB all = 0.5369 GB } __ZipSize{ index = 0.0010 GB value = 0.3297 GB dict = 4.91 MB all = 0.3451 GB } UnZip/Zip{ index = 20.8451 value = 1.4980 dict = 1.00 all = 1.5559 } Zip/UnZip{ index = 0.0480 value = 0.6676 dict = 1.00 all = 0.6427 } ---------------------------- total value len = 68.590768 GB avg = 0.181 KB (by entry num) total key len = 7.316005 GB avg = 0.019 KB total ukey len = 3.453131 GB avg = 0.008 KB total ukey num = 0.422286535 Billion total entry num = 0.379164543 Billion write speed all = 2.899133395 MB/sec (with seq num) write speed all = 2.783280904 MB/sec (without seq num) 2024/08/20-00:50:24.225143 140500515026496 [compaction_job.cc:1944] [default] [JOB 3104] Generated table #14623090: 2684355 keys, 345062987 bytes, temperature: kWarm 2024/08/20-00:50:24.225222 140500515026496 EVENT_LOG_v1 {"time_micros": 1724086224225177, "cf_name": "default", "job": 3104, "event": "table_file_creation", "file_number": 14623090, "file_size": 345062987, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189415896974, "largest_seqno": 189634997918, "table_properties": {"data_size": 329722336, "index_size": 1030208, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 42949680, "raw_average_key_size": 16, "raw_value_size": 493921320, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 2684355, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724086210, "oldest_key_time": 0, "file_creation_time": 1724086210, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14623090, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:50:24.487466 140500515026496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623091.sst) /dev/shm/ToplingTemp/Topling-82985-leySF6 2024/08/20-00:50:24.498992 140500515026496 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623091.sst): first pass time = 0.01's, 0.052'MB/sec 2024/08/20-00:50:24.520473 140500515026496 [top_zip_table_builder.cc:2718] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623091.sst): part-0: reject ZipStore ratio = 1.536 2024/08/20-00:50:24.520751 140500515026496 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623091.sst): old prealloc_size = 590558003, real_size = 1232 2024/08/20-00:50:24.520776 140500515026496 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623091.sst): index type = UintIndex_IL_256_32 , store type = MixedLenBlobStore usrkeys = 3 prefix = 4 raw-key = 0.0000 GB zip-key = 0.0000 GB avg-key = 4.00 avg-zkey = 90.67 raw-val = 0.0000 GB zip-val = 0.0000 GB avg-val = 184.00 avg-zval = 250.67 2024/08/20-00:50:24.521092 140500515026496 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00/14623091.sst): second pass time = 0.02's, 0.027'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.00's, 6.829'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 13.552'MB/sec (index lex order gen) rebuild zvType time = 0.00's, 0.027'MB/sec write SST data time = 0.00's, 8.055'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 0.02's, unzip length = 0.000'GB zip my value throughput = 0.027'MB/sec zip pipeline throughput = 0.027'MB/sec entries = 3 avg-key = 16.00 avg-zkey = 106.67 avg-val = 184.00 avg-zval = 250.67 usrkeys = 3 avg-key = 8.00 avg-zkey = 106.67 avg-val = 191.00 avg-zval = 250.67 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 3 bytes = 32 } seq expand size = 21 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0000 GB value = 0.0000 GB dict = 0.00 MB all = 0.0000 GB } __ZipSize{ index = 0.0000 GB value = 0.0000 GB dict = 0.00 MB all = 0.0000 GB } UnZip/Zip{ index = 0.0750 value = 0.7340 dict = 0.00 all = 0.2648 } Zip/UnZip{ index = 13.3333 value = 1.3623 dict = 0.00 all = 3.7767 } ---------------------------- total value len = 68.590768 GB avg = 0.181 KB (by entry num) total key len = 7.316005 GB avg = 0.019 KB total ukey len = 3.453131 GB avg = 0.008 KB total ukey num = 0.422286538 Billion total entry num = 0.379164546 Billion write speed all = 2.899050909 MB/sec (with seq num) write speed all = 2.783201715 MB/sec (without seq num) 2024/08/20-00:50:24.525629 140500515026496 [compaction_job.cc:1944] [default] [JOB 3104] Generated table #14623091: 3 keys, 2266 bytes, temperature: kWarm 2024/08/20-00:50:24.525696 140500515026496 EVENT_LOG_v1 {"time_micros": 1724086224525655, "cf_name": "default", "job": 3104, "event": "table_file_creation", "file_number": 14623091, "file_size": 2266, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189440986910, "largest_seqno": 189600299815, "table_properties": {"data_size": 752, "index_size": 320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 48, "raw_average_key_size": 16, "raw_value_size": 552, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 3, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-MixedLenBlobStore", "creation_time": 1724086224, "oldest_key_time": 0, "file_creation_time": 1724086224, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14623091, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:50:24.538995 140500515026496 [compaction_job.cc:699] job-03104: subcompact[0], size: 4.272277 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03104/att-00 [14623075,14623076,14623077,14623078,14623079,14623080,14623081,14623082,14623083,14623084,14623085,14623086,14623087,14623088,14623089,14623090,14623091] 2024/08/20-00:50:25.350015 140500515026496 (Original Log Time 2024/08/20-00:47:25.194036) EVENT_LOG_v1 {"time_micros": 1724086045193984, "job": 3104, "event": "compaction_started", "compaction_reason": "ManualCompaction", "files_L4": [14622577, 14622578, 14622579, 14622580, 14622581, 14622582, 14622583, 14622584, 14622585, 14622586, 14622587, 14622588, 14622589, 14622590, 14622591, 14622592, 14622593, 14622594, 14622595, 14622596, 14622671, 14622672, 14622673, 14622674, 14622675, 14622676, 14622677, 14622678], "score": -1, "input_data_size": 4466277569, "oldest_snapshot_seqno": 189658938907} 2024/08/20-00:50:25.350021 140500515026496 (Original Log Time 2024/08/20-00:50:25.348200) [compaction_job.cc:2018] [default] [JOB 3104] Compacted 28@4 files to L5 => 4272277287 bytes 2024/08/20-00:50:25.350024 140500515026496 (Original Log Time 2024/08/20-00:50:25.349839) [compaction_job.cc:1220] [default] compacted to: files[0 0 0 0 0 17 0] max score 0.00, MB/sec: 24.9 rd, 23.8 wr, level 5, files in(28, 0) out(17 +0 blob) MB in(4259.4, 0.0 +0.0 blob) out(4074.4 +0.0 blob), read-write-amplify(2.0) write-amplify(1.0) OK, records in: 32952977, records dropped: 0 output_compression: Snappy 2024/08/20-00:50:25.350030 140500515026496 (Original Log Time 2024/08/20-00:50:25.349980) EVENT_LOG_v1 {"time_micros": 1724086225349918, "cf": "default", "job": 3104, "event": "compaction_finished", "compaction_time_micros": 179344971, "compaction_time_cpu_micros": 59533017, "output_level": 5, "num_output_files": 17, "total_output_size": 4272277287, "num_input_records": 32952977, "num_output_records": 32952977, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 2649177091, "file_range_sync_nanos": 0, "file_fsync_nanos": 12119, "file_prepare_write_nanos": 456076, "lsm_state": [0, 0, 0, 0, 0, 17, 0]} 2024/08/20-00:50:25.489451 140500515026496 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"a007a635-5c0a-4b4a-be96-b9c695cb016b","dbStarts":1724082090,"starts":1724086045,"executesMs":180295,"compactionJobId":3104,"attempt":0,"compactionInputRawBytes":6590595400,"compactionInputZipBytes":4466277569,"compactionOutputRawBytes":0,"compactionOutputZipBytes":4272277287,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-ee203291c15f7cf01710369d4d4413a8-1fe9bcec741d398c-00"}}, response = 2024/08/20-00:50:25.489562 140500515026496 INFO dcompact_worker.cpp:1456: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-08-19T23.41.30/.rocksdb/job-03104/att-00: olev 5, work 180.296 s, result 5.360 ms, install 1.881 ms, input{raw 6.138 GiB zip 4.160 GiB}