2024/07/08-15:12:01.151986 139818973103680 DEBG dcompact_worker.cpp:2211: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00: fork to child time = 0.023090 sec 2024/07/08-15:12:01.152723 139818973103680 DEBG dcompact_worker.cpp:1011: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00 2024/07/08-15:12:01.161555 139818973103680 DEBG dcompact_worker.cpp:1031: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00 2024/07/08-15:12:01.163432 139818973103680 TRAC dcompact_worker.cpp:1081: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/07/08-15:12:01.163776 139818973103680 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/07/08-15:12:01.224239 139818973103680 [version_set.cc:6139] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-07-08T14.41.36/.rocksdb/job-02294/att-00/MANIFEST-12625661 2024/07/08-15:12:01.278942 139818973103680 [version_set.cc:6182] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-07-08T14.41.36/.rocksdb/job-02294/att-00/MANIFEST-12625661 succeeded,manifest_file_number is 12625661, next_file_number is 12626973, last_sequence is 147210097690, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 12626714 2024/07/08-15:12:01.278952 139818973103680 [version_set.cc:6197] Column family [default] (ID 0), log number is 0 2024/07/08-15:12:01.279093 139818973103680 [version_set.cc:5652] Creating manifest 12626973 2024/07/08-15:12:01.307022 139818973103680 DEBG dcompact_worker.cpp:1215: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00: bottommost_level: fake = 1, rpc = 0 2024/07/08-15:12:01.325723 139818973103680 [compaction_job.cc:2433] [default] [JOB 2294] Compacting 10@4 + 16@5 files to L5, score -1.00, subcompactions 1 : 1 2024/07/08-15:12:01.325756 139818973103680 [compaction_job.cc:2441] [default]: Compaction start summary: Base version 1701 Base level 4, inputs: [12625967(172MB) 12625968(172MB) 12625969(172MB) 12625970(172MB) 12625971(172MB) 12625972(172MB) 12625973(173MB) 12625974(172MB) 12625975(64MB) 12625976(172MB)], [12602637(197MB) 12625557(212MB) 12624385(117MB) 12603399(277MB) 12604674(115MB) 12619010(241MB) 12561239(184MB) 12625223(160MB) 12625582(253MB) 12517929(25MB) 12512556(36MB) 12622290(259MB) 12613882(190MB) 12620394(217MB) 12613626(192MB) 12608621(256MB)] 2024/07/08-15:12:01.330031 139818973103680 [top_zip_table.cc:59] core git_version_hash_info_is:commit 7517bad5c1a147474c3830831b9a22c996321ae8 Author: leipeng Date: 2024-01-08 19:18:17 +0800 vfork_cmd: call close before wait_finish g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/07/08-15:12:01.330082 139818973103680 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit 7517bad5c1a147474c3830831b9a22c996321ae8 Author: leipeng Date: 2024-01-08 19:18:17 +0800 vfork_cmd: call close before wait_finish g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/07/08-15:12:01.330119 139818973103680 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit 7517bad5c1a147474c3830831b9a22c996321ae8 Author: leipeng Date: 2024-01-08 19:18:17 +0800 vfork_cmd: call close before wait_finish g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/07/08-15:12:01.330133 139818973103680 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit 7f80399bfbd2071b6e0f5683e8ab7225b8e6e394 Author: leipeng Date: 2024-01-19 14:52:30 +0800 Change ToplingZipTableOptions::sampleRatio to vector g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/07/08-15:12:01.330158 139818973103680 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit 1240987ca13d0fe562c9b9c037155079f0877840 Author: leipeng Date: 2024-01-19 15:07:04 +0800 Change ToplingZipTableOptions::sampleRatio to vector g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/07/08-15:12:05.941977 139818973103680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626974.sst) /dev/shm/ToplingTemp/Topling-13556-MG8AQc 2024/07/08-15:12:18.801457 139818973103680 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626974.sst): first pass time = 12.86's, 41.749'MB/sec 2024/07/08-15:12:24.269322 139818973103680 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626974.sst): old prealloc_size = 590558003, real_size = 330809432 2024/07/08-15:12:24.269407 139818973103680 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626974.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2684355 prefix = 4 raw-key = 0.0107 GB zip-key = 0.0004 GB avg-key = 4.00 avg-zkey = 0.16 raw-val = 0.4939 GB zip-val = 0.3297 GB avg-val = 184.00 avg-zval = 122.83 2024/07/08-15:12:26.548518 139818973103680 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626974.sst): second pass time = 5.47's, 90.366'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.28's, 151.863'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 11106.430'MB/sec (index lex order gen) rebuild zvType time = 1.91's, 0.352'MB/sec write SST data time = 0.37's, 930.549'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.914'MB, throughput = 95727.889'MB/sec zip my value time = 5.47's, unzip length = 0.494'GB zip my value throughput = 90.366'MB/sec zip pipeline throughput = 90.366'MB/sec entries = 2684355 avg-key = 16.00 avg-zkey = 0.16 avg-val = 184.00 avg-zval = 122.83 usrkeys = 2684355 avg-key = 8.00 avg-zkey = 0.16 avg-val = 191.00 avg-zval = 122.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2684355 bytes = 11072976 } seq expand size = 18790485 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0215 GB value = 0.4939 GB dict = 4.91 MB all = 0.5369 GB } __ZipSize{ index = 0.0004 GB value = 0.3297 GB dict = 4.91 MB all = 0.3461 GB } UnZip/Zip{ index = 51.1657 value = 1.4980 dict = 1.00 all = 1.5511 } Zip/UnZip{ index = 0.0195 value = 0.6676 dict = 1.00 all = 0.6447 } ---------------------------- total value len = 0.512712 GB avg = 0.191 KB (by entry num) total key len = 0.042950 GB avg = 0.016 KB total ukey len = 0.021475 GB avg = 0.008 KB total ukey num = 0.002684355 Billion total entry num = 0.002684355 Billion write speed all = 0.029303232 MB/sec (with seq num) write speed all = 0.028170740 MB/sec (without seq num) 2024/07/08-15:12:26.992874 139818973103680 [compaction_job.cc:1944] [default] [JOB 2294] Generated table #12626974: 2684355 keys, 346126347 bytes, temperature: kWarm 2024/07/08-15:12:26.992982 139818973103680 EVENT_LOG_v1 {"time_micros": 1720422746992922, "cf_name": "default", "job": 2294, "event": "table_file_creation", "file_number": 12626974, "file_size": 346126347, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 142287115373, "largest_seqno": 147180687236, "table_properties": {"data_size": 329718512, "index_size": 419712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 42949680, "raw_average_key_size": 16, "raw_value_size": 493921320, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 2684355, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1720422725, "oldest_key_time": 0, "file_creation_time": 1720422725, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12626974, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:12:27.113806 139818973103680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626975.sst) /dev/shm/ToplingTemp/Topling-13556-NxAB5g 2024/07/08-15:12:33.931131 139818973103680 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626975.sst): first pass time = 6.82's, 78.752'MB/sec 2024/07/08-15:12:38.933229 139818973103680 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626975.sst): old prealloc_size = 590558003, real_size = 330804152 2024/07/08-15:12:38.933340 139818973103680 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626975.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2684355 prefix = 4 raw-key = 0.0107 GB zip-key = 0.0004 GB avg-key = 4.00 avg-zkey = 0.16 raw-val = 0.4939 GB zip-val = 0.3297 GB avg-val = 184.00 avg-zval = 122.83 2024/07/08-15:12:41.100260 139818973103680 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626975.sst): second pass time = 5.00's, 98.781'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.17's, 159.720'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 12929.731'MB/sec (index lex order gen) rebuild zvType time = 1.79's, 0.374'MB/sec write SST data time = 0.37's, 924.396'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.914'MB, throughput = 103135.775'MB/sec zip my value time = 5.00's, unzip length = 0.494'GB zip my value throughput = 98.781'MB/sec zip pipeline throughput = 98.781'MB/sec entries = 2684355 avg-key = 16.00 avg-zkey = 0.16 avg-val = 184.00 avg-zval = 122.83 usrkeys = 2684355 avg-key = 8.00 avg-zkey = 0.16 avg-val = 191.00 avg-zval = 122.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2684355 bytes = 11072976 } seq expand size = 18790485 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0215 GB value = 0.4939 GB dict = 4.91 MB all = 0.5369 GB } __ZipSize{ index = 0.0004 GB value = 0.3297 GB dict = 4.91 MB all = 0.3461 GB } UnZip/Zip{ index = 51.1657 value = 1.4980 dict = 1.00 all = 1.5511 } Zip/UnZip{ index = 0.0195 value = 0.6675 dict = 1.00 all = 0.6447 } ---------------------------- total value len = 1.025424 GB avg = 0.191 KB (by entry num) total key len = 0.085899 GB avg = 0.016 KB total ukey len = 0.042950 GB avg = 0.008 KB total ukey num = 0.005368710 Billion total entry num = 0.005368710 Billion write speed all = 0.058561524 MB/sec (with seq num) write speed all = 0.056298277 MB/sec (without seq num) 2024/07/08-15:12:41.484827 139818973103680 [compaction_job.cc:1944] [default] [JOB 2294] Generated table #12626975: 2684355 keys, 346121035 bytes, temperature: kWarm 2024/07/08-15:12:41.484899 139818973103680 EVENT_LOG_v1 {"time_micros": 1720422761484864, "cf_name": "default", "job": 2294, "event": "table_file_creation", "file_number": 12626975, "file_size": 346121035, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 142213868559, "largest_seqno": 147180686770, "table_properties": {"data_size": 329713232, "index_size": 419712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 42949680, "raw_average_key_size": 16, "raw_value_size": 493921320, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 2684355, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1720422747, "oldest_key_time": 0, "file_creation_time": 1720422747, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12626975, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:12:41.581074 139818973103680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626976.sst) /dev/shm/ToplingTemp/Topling-13556-YUPNTf 2024/07/08-15:12:50.955684 139818973103680 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626976.sst): first pass time = 9.37's, 57.269'MB/sec 2024/07/08-15:12:55.978267 139818973103680 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626976.sst): old prealloc_size = 590558003, real_size = 330801528 2024/07/08-15:12:55.978360 139818973103680 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626976.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2684355 prefix = 4 raw-key = 0.0107 GB zip-key = 0.0004 GB avg-key = 4.00 avg-zkey = 0.16 raw-val = 0.4939 GB zip-val = 0.3297 GB avg-val = 184.00 avg-zval = 122.83 2024/07/08-15:12:58.155742 139818973103680 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626976.sst): second pass time = 5.02's, 98.382'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.18's, 158.955'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 12715.848'MB/sec (index lex order gen) rebuild zvType time = 1.81's, 0.371'MB/sec write SST data time = 0.37's, 944.098'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.914'MB, throughput = 109321.765'MB/sec zip my value time = 5.02's, unzip length = 0.494'GB zip my value throughput = 98.382'MB/sec zip pipeline throughput = 98.382'MB/sec entries = 2684355 avg-key = 16.00 avg-zkey = 0.16 avg-val = 184.00 avg-zval = 122.83 usrkeys = 2684355 avg-key = 8.00 avg-zkey = 0.16 avg-val = 191.00 avg-zval = 122.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2684355 bytes = 11072976 } seq expand size = 18790485 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0215 GB value = 0.4939 GB dict = 4.91 MB all = 0.5369 GB } __ZipSize{ index = 0.0004 GB value = 0.3297 GB dict = 4.91 MB all = 0.3461 GB } UnZip/Zip{ index = 51.1657 value = 1.4980 dict = 1.00 all = 1.5511 } Zip/UnZip{ index = 0.0195 value = 0.6675 dict = 1.00 all = 0.6447 } ---------------------------- total value len = 1.538135 GB avg = 0.191 KB (by entry num) total key len = 0.128849 GB avg = 0.016 KB total ukey len = 0.064425 GB avg = 0.008 KB total ukey num = 0.008053065 Billion total entry num = 0.008053065 Billion write speed all = 0.087763409 MB/sec (with seq num) write speed all = 0.084371587 MB/sec (without seq num) 2024/07/08-15:12:58.533227 139818973103680 [compaction_job.cc:1944] [default] [JOB 2294] Generated table #12626976: 2684355 keys, 346118411 bytes, temperature: kWarm 2024/07/08-15:12:58.533309 139818973103680 EVENT_LOG_v1 {"time_micros": 1720422778533264, "cf_name": "default", "job": 2294, "event": "table_file_creation", "file_number": 12626976, "file_size": 346118411, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 142267758729, "largest_seqno": 147180687240, "table_properties": {"data_size": 329710608, "index_size": 419712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 42949680, "raw_average_key_size": 16, "raw_value_size": 493921320, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 2684355, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1720422761, "oldest_key_time": 0, "file_creation_time": 1720422761, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12626976, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:12:58.683771 139818973103680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626977.sst) /dev/shm/ToplingTemp/Topling-13556-pmkFtg 2024/07/08-15:13:06.582393 139818973103680 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626977.sst): first pass time = 7.90's, 67.971'MB/sec 2024/07/08-15:13:11.601646 139818973103680 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626977.sst): old prealloc_size = 590558003, real_size = 330811640 2024/07/08-15:13:11.601726 139818973103680 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626977.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2684355 prefix = 4 raw-key = 0.0107 GB zip-key = 0.0004 GB avg-key = 4.00 avg-zkey = 0.16 raw-val = 0.4939 GB zip-val = 0.3297 GB avg-val = 184.00 avg-zval = 122.83 2024/07/08-15:13:13.777625 139818973103680 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626977.sst): second pass time = 5.02's, 98.443'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.18's, 159.068'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 13964.798'MB/sec (index lex order gen) rebuild zvType time = 1.81's, 0.371'MB/sec write SST data time = 0.37's, 938.628'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.914'MB, throughput = 93564.310'MB/sec zip my value time = 5.02's, unzip length = 0.494'GB zip my value throughput = 98.443'MB/sec zip pipeline throughput = 98.443'MB/sec entries = 2684355 avg-key = 16.00 avg-zkey = 0.16 avg-val = 184.00 avg-zval = 122.83 usrkeys = 2684355 avg-key = 8.00 avg-zkey = 0.16 avg-val = 191.00 avg-zval = 122.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2684355 bytes = 11072976 } seq expand size = 18790485 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0215 GB value = 0.4939 GB dict = 4.91 MB all = 0.5369 GB } __ZipSize{ index = 0.0004 GB value = 0.3297 GB dict = 4.91 MB all = 0.3461 GB } UnZip/Zip{ index = 51.1657 value = 1.4980 dict = 1.00 all = 1.5511 } Zip/UnZip{ index = 0.0195 value = 0.6676 dict = 1.00 all = 0.6447 } ---------------------------- total value len = 2.050847 GB avg = 0.191 KB (by entry num) total key len = 0.171799 GB avg = 0.016 KB total ukey len = 0.085899 GB avg = 0.008 KB total ukey num = 0.010737420 Billion total entry num = 0.010737420 Billion write speed all = 0.116921716 MB/sec (with seq num) write speed all = 0.112403002 MB/sec (without seq num) 2024/07/08-15:13:14.155248 139818973103680 [compaction_job.cc:1944] [default] [JOB 2294] Generated table #12626977: 2684355 keys, 346128523 bytes, temperature: kWarm 2024/07/08-15:13:14.155327 139818973103680 EVENT_LOG_v1 {"time_micros": 1720422794155284, "cf_name": "default", "job": 2294, "event": "table_file_creation", "file_number": 12626977, "file_size": 346128523, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 142173176304, "largest_seqno": 147180687239, "table_properties": {"data_size": 329720720, "index_size": 419712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 42949680, "raw_average_key_size": 16, "raw_value_size": 493921320, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 2684355, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1720422778, "oldest_key_time": 0, "file_creation_time": 1720422778, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12626977, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:13:14.256918 139818973103680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626978.sst) /dev/shm/ToplingTemp/Topling-13556-1kPc6g 2024/07/08-15:13:21.611778 139818973103680 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626978.sst): first pass time = 7.35's, 72.996'MB/sec 2024/07/08-15:13:26.617518 139818973103680 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626978.sst): old prealloc_size = 590558003, real_size = 330798264 2024/07/08-15:13:26.617608 139818973103680 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626978.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2684355 prefix = 4 raw-key = 0.0107 GB zip-key = 0.0004 GB avg-key = 4.00 avg-zkey = 0.16 raw-val = 0.4939 GB zip-val = 0.3297 GB avg-val = 184.00 avg-zval = 122.83 2024/07/08-15:13:28.802790 139818973103680 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626978.sst): second pass time = 5.00's, 98.710'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.19's, 158.386'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 11245.097'MB/sec (index lex order gen) rebuild zvType time = 1.81's, 0.370'MB/sec write SST data time = 0.37's, 928.407'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.914'MB, throughput = 101242.459'MB/sec zip my value time = 5.00's, unzip length = 0.494'GB zip my value throughput = 98.710'MB/sec zip pipeline throughput = 98.710'MB/sec entries = 2684355 avg-key = 16.00 avg-zkey = 0.16 avg-val = 184.00 avg-zval = 122.83 usrkeys = 2684355 avg-key = 8.00 avg-zkey = 0.16 avg-val = 191.00 avg-zval = 122.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2684355 bytes = 11072976 } seq expand size = 18790485 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0215 GB value = 0.4939 GB dict = 4.91 MB all = 0.5369 GB } __ZipSize{ index = 0.0004 GB value = 0.3297 GB dict = 4.91 MB all = 0.3461 GB } UnZip/Zip{ index = 51.1657 value = 1.4981 dict = 1.00 all = 1.5511 } Zip/UnZip{ index = 0.0195 value = 0.6675 dict = 1.00 all = 0.6447 } ---------------------------- total value len = 2.563559 GB avg = 0.191 KB (by entry num) total key len = 0.214748 GB avg = 0.016 KB total ukey len = 0.107374 GB avg = 0.008 KB total ukey num = 0.013421775 Billion total entry num = 0.013421775 Billion write speed all = 0.146036718 MB/sec (with seq num) write speed all = 0.140392787 MB/sec (without seq num) 2024/07/08-15:13:29.187563 139818973103680 [compaction_job.cc:1944] [default] [JOB 2294] Generated table #12626978: 2684355 keys, 346115147 bytes, temperature: kWarm 2024/07/08-15:13:29.187635 139818973103680 EVENT_LOG_v1 {"time_micros": 1720422809187601, "cf_name": "default", "job": 2294, "event": "table_file_creation", "file_number": 12626978, "file_size": 346115147, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 142249876106, "largest_seqno": 147180687235, "table_properties": {"data_size": 329707344, "index_size": 419712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 42949680, "raw_average_key_size": 16, "raw_value_size": 493921320, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 2684355, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1720422794, "oldest_key_time": 0, "file_creation_time": 1720422794, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12626978, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:13:29.292874 139818973103680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626979.sst) /dev/shm/ToplingTemp/Topling-13556-kuMmjg 2024/07/08-15:13:38.520279 139818973103680 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626979.sst): first pass time = 9.23's, 58.183'MB/sec 2024/07/08-15:13:46.960380 139818973103680 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626979.sst): old prealloc_size = 590558003, real_size = 330809656 2024/07/08-15:13:46.960453 139818973103680 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626979.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2684355 prefix = 4 raw-key = 0.0107 GB zip-key = 0.0004 GB avg-key = 4.00 avg-zkey = 0.16 raw-val = 0.4939 GB zip-val = 0.3297 GB avg-val = 184.00 avg-zval = 122.83 2024/07/08-15:13:49.501894 139818973103680 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626979.sst): second pass time = 8.44's, 58.535'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.54's, 136.190'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 15034.280'MB/sec (index lex order gen) rebuild zvType time = 2.16's, 0.311'MB/sec write SST data time = 0.38's, 908.861'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.914'MB, throughput = 106456.032'MB/sec zip my value time = 8.44's, unzip length = 0.494'GB zip my value throughput = 58.535'MB/sec zip pipeline throughput = 118.716'MB/sec entries = 2684355 avg-key = 16.00 avg-zkey = 0.16 avg-val = 184.00 avg-zval = 122.83 usrkeys = 2684355 avg-key = 8.00 avg-zkey = 0.16 avg-val = 191.00 avg-zval = 122.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2684355 bytes = 11072976 } seq expand size = 18790485 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0215 GB value = 0.4939 GB dict = 4.91 MB all = 0.5369 GB } __ZipSize{ index = 0.0004 GB value = 0.3297 GB dict = 4.91 MB all = 0.3461 GB } UnZip/Zip{ index = 51.1657 value = 1.4980 dict = 1.00 all = 1.5511 } Zip/UnZip{ index = 0.0195 value = 0.6676 dict = 1.00 all = 0.6447 } ---------------------------- total value len = 3.076271 GB avg = 0.191 KB (by entry num) total key len = 0.257698 GB avg = 0.016 KB total ukey len = 0.128849 GB avg = 0.008 KB total ukey num = 0.016106130 Billion total entry num = 0.016106130 Billion write speed all = 0.175053601 MB/sec (with seq num) write speed all = 0.168288245 MB/sec (without seq num) 2024/07/08-15:13:49.914411 139818973103680 [compaction_job.cc:1944] [default] [JOB 2294] Generated table #12626979: 2684355 keys, 346126539 bytes, temperature: kWarm 2024/07/08-15:13:49.914494 139818973103680 EVENT_LOG_v1 {"time_micros": 1720422829914449, "cf_name": "default", "job": 2294, "event": "table_file_creation", "file_number": 12626979, "file_size": 346126539, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 142372510937, "largest_seqno": 147180687233, "table_properties": {"data_size": 329718736, "index_size": 419712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 42949680, "raw_average_key_size": 16, "raw_value_size": 493921320, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 2684355, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1720422809, "oldest_key_time": 0, "file_creation_time": 1720422809, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12626979, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:13:50.063339 139818973103680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626980.sst) /dev/shm/ToplingTemp/Topling-13556-ztRLdg 2024/07/08-15:13:58.714376 139818973103680 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626980.sst): first pass time = 8.65's, 62.059'MB/sec 2024/07/08-15:14:03.739749 139818973103680 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626980.sst): old prealloc_size = 590558003, real_size = 330822616 2024/07/08-15:14:03.739826 139818973103680 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626980.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2684355 prefix = 4 raw-key = 0.0107 GB zip-key = 0.0004 GB avg-key = 4.00 avg-zkey = 0.16 raw-val = 0.4939 GB zip-val = 0.3297 GB avg-val = 184.00 avg-zval = 122.83 2024/07/08-15:14:05.912516 139818973103680 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626980.sst): second pass time = 5.02's, 98.325'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.17's, 159.309'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 13473.468'MB/sec (index lex order gen) rebuild zvType time = 1.80's, 0.372'MB/sec write SST data time = 0.37's, 934.356'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.914'MB, throughput = 99419.415'MB/sec zip my value time = 5.02's, unzip length = 0.494'GB zip my value throughput = 98.325'MB/sec zip pipeline throughput = 98.325'MB/sec entries = 2684355 avg-key = 16.00 avg-zkey = 0.16 avg-val = 184.00 avg-zval = 122.83 usrkeys = 2684355 avg-key = 8.00 avg-zkey = 0.16 avg-val = 191.00 avg-zval = 122.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2684355 bytes = 11072976 } seq expand size = 18790485 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0215 GB value = 0.4939 GB dict = 4.91 MB all = 0.5369 GB } __ZipSize{ index = 0.0004 GB value = 0.3297 GB dict = 4.91 MB all = 0.3461 GB } UnZip/Zip{ index = 51.1657 value = 1.4979 dict = 1.00 all = 1.5510 } Zip/UnZip{ index = 0.0195 value = 0.6676 dict = 1.00 all = 0.6447 } ---------------------------- total value len = 3.588983 GB avg = 0.191 KB (by entry num) total key len = 0.300648 GB avg = 0.016 KB total ukey len = 0.150324 GB avg = 0.008 KB total ukey num = 0.018790485 Billion total entry num = 0.018790485 Billion write speed all = 0.204053378 MB/sec (with seq num) write speed all = 0.196167257 MB/sec (without seq num) 2024/07/08-15:14:06.315128 139818973103680 [compaction_job.cc:1944] [default] [JOB 2294] Generated table #12626980: 2684355 keys, 346139531 bytes, temperature: kWarm 2024/07/08-15:14:06.315214 139818973103680 EVENT_LOG_v1 {"time_micros": 1720422846315161, "cf_name": "default", "job": 2294, "event": "table_file_creation", "file_number": 12626980, "file_size": 346139531, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 142405485986, "largest_seqno": 147180686767, "table_properties": {"data_size": 329731696, "index_size": 419712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 42949680, "raw_average_key_size": 16, "raw_value_size": 493921320, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 2684355, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1720422830, "oldest_key_time": 0, "file_creation_time": 1720422830, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12626980, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:14:06.411461 139818973103680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626981.sst) /dev/shm/ToplingTemp/Topling-13556-KAT8le 2024/07/08-15:14:10.430144 139818973103680 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626981.sst): first pass time = 4.02's, 60.194'MB/sec 2024/07/08-15:14:12.746711 139818973103680 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626981.sst): old prealloc_size = 590558003, real_size = 157525856 2024/07/08-15:14:12.746787 139818973103680 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626981.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 1209482 prefix = 4 raw-key = 0.0048 GB zip-key = 0.0002 GB avg-key = 4.00 avg-zkey = 0.16 raw-val = 0.2225 GB zip-val = 0.1570 GB avg-val = 184.00 avg-zval = 129.84 2024/07/08-15:14:13.701366 139818973103680 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626981.sst): second pass time = 2.32's, 96.122'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 172.236'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 6566.323'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.387'MB/sec write SST data time = 0.17's, 951.798'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 2.211'MB, throughput = 46575.606'MB/sec zip my value time = 2.32's, unzip length = 0.223'GB zip my value throughput = 96.122'MB/sec zip pipeline throughput = 96.122'MB/sec entries = 1209482 avg-key = 16.00 avg-zkey = 0.16 avg-val = 184.00 avg-zval = 129.84 usrkeys = 1209482 avg-key = 8.00 avg-zkey = 0.16 avg-val = 191.00 avg-zval = 129.84 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1209482 bytes = 4989136 } seq expand size = 8466374 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0097 GB value = 0.2225 GB dict = 2.21 MB all = 0.2419 GB } __ZipSize{ index = 0.0002 GB value = 0.1570 GB dict = 2.21 MB all = 0.1644 GB } UnZip/Zip{ index = 51.1279 value = 1.4172 dict = 1.00 all = 1.4712 } Zip/UnZip{ index = 0.0196 value = 0.7056 dict = 1.00 all = 0.6797 } ---------------------------- total value len = 3.819994 GB avg = 0.191 KB (by entry num) total key len = 0.319999 GB avg = 0.016 KB total ukey len = 0.160000 GB avg = 0.008 KB total ukey num = 0.019999967 Billion total entry num = 0.019999967 Billion write speed all = 0.217098917 MB/sec (with seq num) write speed all = 0.208708620 MB/sec (without seq num) 2024/07/08-15:14:13.881762 139818973103680 [compaction_job.cc:1944] [default] [JOB 2294] Generated table #12626981: 1209482 keys, 164424661 bytes, temperature: kWarm 2024/07/08-15:14:13.881830 139818973103680 EVENT_LOG_v1 {"time_micros": 1720422853881796, "cf_name": "default", "job": 2294, "event": "table_file_creation", "file_number": 12626981, "file_size": 164424661, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 142344435022, "largest_seqno": 147180687238, "table_properties": {"data_size": 157034080, "index_size": 189248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 19351712, "raw_average_key_size": 16, "raw_value_size": 222544688, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 1209482, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1720422846, "oldest_key_time": 0, "file_creation_time": 1720422846, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12626981, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:14:13.935265 139818973103680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626982.sst) /dev/shm/ToplingTemp/Topling-13556-oS1WLf 2024/07/08-15:14:19.124822 139818973103680 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626982.sst): first pass time = 5.19's, 103.454'MB/sec 2024/07/08-15:14:24.477080 139818973103680 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626982.sst): old prealloc_size = 590558003, real_size = 330804968 2024/07/08-15:14:24.477164 139818973103680 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626982.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2684355 prefix = 4 raw-key = 0.0107 GB zip-key = 0.0004 GB avg-key = 4.00 avg-zkey = 0.16 raw-val = 0.4939 GB zip-val = 0.3297 GB avg-val = 184.00 avg-zval = 122.83 2024/07/08-15:14:26.658884 139818973103680 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626982.sst): second pass time = 5.35's, 92.330'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.18's, 158.641'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 13559.217'MB/sec (index lex order gen) rebuild zvType time = 1.81's, 0.371'MB/sec write SST data time = 0.37's, 924.159'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.914'MB, throughput = 117386.207'MB/sec zip my value time = 5.35's, unzip length = 0.494'GB zip my value throughput = 92.330'MB/sec zip pipeline throughput = 92.330'MB/sec entries = 2684355 avg-key = 16.00 avg-zkey = 0.16 avg-val = 184.00 avg-zval = 122.83 usrkeys = 2684355 avg-key = 8.00 avg-zkey = 0.16 avg-val = 191.00 avg-zval = 122.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2684355 bytes = 11072976 } seq expand size = 18790485 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0215 GB value = 0.4939 GB dict = 4.91 MB all = 0.5369 GB } __ZipSize{ index = 0.0004 GB value = 0.3297 GB dict = 4.91 MB all = 0.3461 GB } UnZip/Zip{ index = 51.1657 value = 1.4980 dict = 1.00 all = 1.5511 } Zip/UnZip{ index = 0.0195 value = 0.6675 dict = 1.00 all = 0.6447 } ---------------------------- total value len = 4.332706 GB avg = 0.191 KB (by entry num) total key len = 0.362949 GB avg = 0.016 KB total ukey len = 0.181475 GB avg = 0.008 KB total ukey num = 0.022684322 Billion total entry num = 0.022684322 Billion write speed all = 0.246070292 MB/sec (with seq num) write speed all = 0.236560329 MB/sec (without seq num) 2024/07/08-15:14:27.039329 139818973103680 [compaction_job.cc:1944] [default] [JOB 2294] Generated table #12626982: 2684355 keys, 346121867 bytes, temperature: kWarm 2024/07/08-15:14:27.039401 139818973103680 EVENT_LOG_v1 {"time_micros": 1720422867039366, "cf_name": "default", "job": 2294, "event": "table_file_creation", "file_number": 12626982, "file_size": 346121867, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 142186179574, "largest_seqno": 147180687698, "table_properties": {"data_size": 329714048, "index_size": 419712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 42949680, "raw_average_key_size": 16, "raw_value_size": 493921320, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 2684355, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1720422853, "oldest_key_time": 0, "file_creation_time": 1720422853, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12626982, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:14:27.140422 139818973103680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626983.sst) /dev/shm/ToplingTemp/Topling-13556-KROCFf 2024/07/08-15:14:27.909676 139818973103680 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626983.sst): first pass time = 0.77's, 231.152'MB/sec 2024/07/08-15:14:29.476897 139818973103680 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626983.sst): old prealloc_size = 590558003, real_size = 118443584 2024/07/08-15:14:29.476960 139818973103680 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626983.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 889011 prefix = 4 raw-key = 0.0036 GB zip-key = 0.0001 GB avg-key = 4.00 avg-zkey = 0.16 raw-val = 0.1636 GB zip-val = 0.1181 GB avg-val = 184.00 avg-zval = 132.82 2024/07/08-15:14:30.181635 139818973103680 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00/12626983.sst): second pass time = 1.57's, 104.447'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.70's, 175.283'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 4490.757'MB/sec (index lex order gen) rebuild zvType time = 0.57's, 0.387'MB/sec write SST data time = 0.13's, 947.088'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.637'MB, throughput = 37057.465'MB/sec zip my value time = 1.57's, unzip length = 0.164'GB zip my value throughput = 104.447'MB/sec zip pipeline throughput = 104.447'MB/sec entries = 889011 avg-key = 16.00 avg-zkey = 0.16 avg-val = 184.00 avg-zval = 132.82 usrkeys = 889011 avg-key = 8.00 avg-zkey = 0.16 avg-val = 191.00 avg-zval = 132.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 889011 bytes = 3667184 } seq expand size = 6223077 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0071 GB value = 0.1636 GB dict = 1.64 MB all = 0.1778 GB } __ZipSize{ index = 0.0001 GB value = 0.1181 GB dict = 1.64 MB all = 0.1235 GB } UnZip/Zip{ index = 51.0926 value = 1.3853 dict = 1.00 all = 1.4394 } Zip/UnZip{ index = 0.0196 value = 0.7219 dict = 1.00 all = 0.6947 } ---------------------------- total value len = 4.502507 GB avg = 0.191 KB (by entry num) total key len = 0.377173 GB avg = 0.016 KB total ukey len = 0.188587 GB avg = 0.008 KB total ukey num = 0.023573333 Billion total entry num = 0.023573333 Billion write speed all = 0.255666724 MB/sec (with seq num) write speed all = 0.245785884 MB/sec (without seq num) 2024/07/08-15:14:30.325667 139818973103680 [compaction_job.cc:1944] [default] [JOB 2294] Generated table #12626983: 889011 keys, 123526300 bytes, temperature: kWarm 2024/07/08-15:14:30.325739 139818973103680 EVENT_LOG_v1 {"time_micros": 1720422870325703, "cf_name": "default", "job": 2294, "event": "table_file_creation", "file_number": 12626983, "file_size": 123526300, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 142140768186, "largest_seqno": 146800555810, "table_properties": {"data_size": 118082016, "index_size": 139200, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 14224176, "raw_average_key_size": 16, "raw_value_size": 163578024, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 889011, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1720422867, "oldest_key_time": 0, "file_creation_time": 1720422867, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12626983, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:14:30.362715 139818973103680 [compaction_job.cc:699] job-02294: subcompact[0], size: 3.056948 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02294/att-00 [12626974,12626975,12626976,12626977,12626978,12626979,12626980,12626981,12626982,12626983] 2024/07/08-15:14:30.663311 139818973103680 (Original Log Time 2024/07/08-15:12:01.325813) EVENT_LOG_v1 {"time_micros": 1720422721325771, "job": 2294, "event": "compaction_started", "compaction_reason": "ManualCompaction", "files_L4": [12625967, 12625968, 12625969, 12625970, 12625971, 12625972, 12625973, 12625974, 12625975, 12625976], "files_L5": [12602637, 12625557, 12624385, 12603399, 12604674, 12619010, 12561239, 12625223, 12625582, 12517929, 12512556, 12622290, 12613882, 12620394, 12613626, 12608621], "score": -1, "input_data_size": 4779305810, "oldest_snapshot_seqno": 147210095614} 2024/07/08-15:14:30.663317 139818973103680 (Original Log Time 2024/07/08-15:14:30.662284) [compaction_job.cc:2018] [default] [JOB 2294] Compacted 10@4 + 16@5 files to L5 => 3056948361 bytes 2024/07/08-15:14:30.663318 139818973103680 (Original Log Time 2024/07/08-15:14:30.663219) [compaction_job.cc:1220] [default] compacted to: files[0 0 0 0 0 10 0] max score 0.00, MB/sec: 32.1 rd, 20.5 wr, level 5, files in(10, 16) out(10 +0 blob) MB in(1619.2, 2938.7 +0.0 blob) out(2915.3 +0.0 blob), read-write-amplify(4.6) write-amplify(1.8) OK, records in: 35639973, records dropped: 12066640 output_compression: Snappy 2024/07/08-15:14:30.663322 139818973103680 (Original Log Time 2024/07/08-15:14:30.663271) EVENT_LOG_v1 {"time_micros": 1720422870663248, "cf": "default", "job": 2294, "event": "compaction_finished", "compaction_time_micros": 149036914, "compaction_time_cpu_micros": 58470790, "output_level": 5, "num_output_files": 10, "total_output_size": 3056948361, "num_input_records": 35639973, "num_output_records": 23573333, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 1778767084, "file_range_sync_nanos": 0, "file_fsync_nanos": 7985, "file_prepare_write_nanos": 37644, "lsm_state": [0, 0, 0, 0, 0, 10, 0]} 2024/07/08-15:14:30.805530 139818973103680 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"a007a635-5c0a-4b4a-be96-b9c695cb016b","dbStarts":1720420896,"starts":1720422721,"executesMs":149504,"compactionJobId":2294,"attempt":0,"compactionInputRawBytes":7127994600,"compactionInputZipBytes":4779305810,"compactionOutputRawBytes":0,"compactionOutputZipBytes":3056948361,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-f1d595447084eea020e68d30d9a47bdb-a01bbc9888e381e5-00"}}, response = 2024/07/08-15:14:30.805622 139818973103680 INFO dcompact_worker.cpp:1456: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-07-08T14.41.36/.rocksdb/job-02294/att-00: olev 5, work 149.505 s, result 5.342 ms, install 1.059 ms, input{raw 6.638 GiB zip 4.451 GiB}