2024/11/24-03:27:03.185163 139957424981568 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00: fork to child time = 0.008586 sec 2024/11/24-03:27:03.185912 139957424981568 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00 2024/11/24-03:27:03.192042 139957424981568 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00 2024/11/24-03:27:03.193347 139957424981568 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-03:27:03.193644 139957424981568 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-03:27:03.245810 139957424981568 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-40789/att-00/MANIFEST-206347 2024/11/24-03:27:03.273878 139957424981568 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-40789/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 290303, last_sequence is 4417306414, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 290169 2024/11/24-03:27:03.273886 139957424981568 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-03:27:03.274051 139957424981568 [version_set.cc:5721] Creating manifest 290303 2024/11/24-03:27:03.307358 139957424981568 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-03:27:03.327472 139957424981568 [compaction_job.cc:2437] [default] [JOB 40789] Compacting 1@2 + 19@3 files to L3, score 0.93, subcompactions 1 : 1 2024/11/24-03:27:03.327491 139957424981568 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 30546 Base level 2, inputs: [290300(27MB)], [290264(31MB) 290265(33MB) 290266(30MB) 290267(34MB) 290268(33MB) 290269(33MB) 290270(34MB) 290271(33MB) 290272(35MB) 290273(33MB) 290274(35MB) 290275(13MB) 290234(34MB) 290235(33MB) 290236(35MB) 290237(33MB) 290238(35MB) 290239(33MB) 290240(10MB)] 2024/11/24-03:27:03.334240 139957424981568 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-03:27:03.334270 139957424981568 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-03:27:03.334288 139957424981568 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-03:27:03.334294 139957424981568 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-03:27:03.334304 139957424981568 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-03:27:03.849936 139957424981568 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290304.sst) /dev/shm/ToplingTemp/Topling-258257-uFGaMl 2024/11/24-03:27:04.478858 139957424981568 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290304.sst): first pass time = 0.63's, 213.422'MB/sec 2024/11/24-03:27:07.653788 139957424981568 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290304.sst): old prealloc_size = 147639500, real_size = 29002256 2024/11/24-03:27:07.654537 139957424981568 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290304.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 943258 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1078 GB zip-val = 0.0270 GB avg-val = 114.29 avg-zval = 28.59 2024/11/24-03:27:07.655462 139957424981568 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290304.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:27:08.928081 139957424981568 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290304.sst): second pass time = 3.17's, 33.966'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.27's, 26.013'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2470.240'MB/sec (index lex order gen) rebuild zvType time = 1.12's, 0.211'MB/sec write SST data time = 0.16's, 212.499'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.074'MB, throughput = 35809.667'MB/sec zip my value time = 3.17's, unzip length = 0.108'GB zip my value throughput = 33.966'MB/sec zip pipeline throughput = 90.049'MB/sec entries = 943258 avg-key = 28.00 avg-zkey = 1.90 avg-val = 114.29 avg-zval = 28.59 usrkeys = 943258 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.29 avg-zval = 28.59 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 943258 bytes = 3301424 } seq expand size = 6602806 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0270 GB dict = 1.07 MB all = 0.0331 GB } UnZip/Zip{ index = 10.5124 value = 3.9970 dict = 1.00 all = 4.0490 } Zip/UnZip{ index = 0.0951 value = 0.2502 dict = 1.00 all = 0.2470 } ---------------------------- total value len = 2251.439983 GB avg = 0.200 KB (by entry num) total key len = 306.909374 GB avg = 0.027 KB total ukey len = 237.373592 GB avg = 0.015 KB total ukey num = 15.739604421 Billion total entry num = 11.280918369 Billion write speed all = 38.346453604 MB/sec (with seq num) write speed all = 36.993758869 MB/sec (without seq num) 2024/11/24-03:27:08.971092 139957424981568 [compaction_job.cc:1948] [default] [JOB 40789] Generated table #290304: 943258 keys, 33148034 bytes, temperature: kWarm 2024/11/24-03:27:08.971205 139957424981568 EVENT_LOG_v1 {"time_micros": 1732390028971131, "cf_name": "default", "job": 40789, "event": "table_file_creation", "file_number": 290304, "file_size": 33148034, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4417292278, "table_properties": {"data_size": 26971712, "index_size": 1794560, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26411224, "raw_average_key_size": 28, "raw_value_size": 107806508, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 943258, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390023, "oldest_key_time": 0, "file_creation_time": 1732390023, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290304, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:27:09.025783 139957424981568 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290305.sst) /dev/shm/ToplingTemp/Topling-258257-q3rDEo 2024/11/24-03:27:10.415303 139957424981568 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290305.sst): first pass time = 1.39's, 96.595'MB/sec 2024/11/24-03:27:14.175552 139957424981568 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290305.sst): old prealloc_size = 147639500, real_size = 31515072 2024/11/24-03:27:14.176156 139957424981568 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290305.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919768 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 117.93 avg-zval = 32.09 2024/11/24-03:27:14.176913 139957424981568 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290305.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:27:15.004399 139957424981568 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290305.sst): second pass time = 3.76's, 28.853'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.945'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3157.915'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.329'MB/sec write SST data time = 0.13's, 274.205'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 31323.709'MB/sec zip my value time = 3.76's, unzip length = 0.108'GB zip my value throughput = 28.853'MB/sec zip pipeline throughput = 106.882'MB/sec entries = 919768 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.93 avg-zval = 32.09 usrkeys = 919768 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.93 avg-zval = 32.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919768 bytes = 3219200 } seq expand size = 6438376 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.08 MB all = 0.0356 GB } UnZip/Zip{ index = 10.3764 value = 3.6752 dict = 1.00 all = 3.7709 } Zip/UnZip{ index = 0.0964 value = 0.2721 dict = 1.00 all = 0.2652 } ---------------------------- total value len = 2251.554885 GB avg = 0.200 KB (by entry num) total key len = 306.935127 GB avg = 0.027 KB total ukey len = 237.391988 GB avg = 0.015 KB total ukey num = 15.740524189 Billion total entry num = 11.281838137 Billion write speed all = 38.345069541 MB/sec (with seq num) write speed all = 36.992387715 MB/sec (without seq num) 2024/11/24-03:27:15.051339 139957424981568 [compaction_job.cc:1948] [default] [JOB 40789] Generated table #290305: 919768 keys, 35593202 bytes, temperature: kWarm 2024/11/24-03:27:15.051466 139957424981568 EVENT_LOG_v1 {"time_micros": 1732390035051373, "cf_name": "default", "job": 40789, "event": "table_file_creation", "file_number": 290305, "file_size": 35593202, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4275273701, "largest_seqno": 4417291196, "table_properties": {"data_size": 29512176, "index_size": 1772800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25753504, "raw_average_key_size": 28, "raw_value_size": 108464228, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 919768, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390029, "oldest_key_time": 0, "file_creation_time": 1732390029, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290305, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:27:15.092107 139957424981568 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290306.sst) /dev/shm/ToplingTemp/Topling-258257-BN1HSp 2024/11/24-03:27:16.216312 139957424981568 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290306.sst): first pass time = 1.12's, 119.394'MB/sec 2024/11/24-03:27:18.028692 139957424981568 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290306.sst): old prealloc_size = 147639500, real_size = 27714640 2024/11/24-03:27:18.029261 139957424981568 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290306.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 945196 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1078 GB zip-val = 0.0257 GB avg-val = 114.00 avg-zval = 27.16 2024/11/24-03:27:18.030013 139957424981568 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290306.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:27:18.969274 139957424981568 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290306.sst): second pass time = 1.81's, 59.487'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 33.753'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3303.647'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.333'MB/sec write SST data time = 0.23's, 138.366'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.073'MB, throughput = 37209.651'MB/sec zip my value time = 1.81's, unzip length = 0.108'GB zip my value throughput = 59.487'MB/sec zip pipeline throughput = 72.185'MB/sec entries = 945196 avg-key = 28.00 avg-zkey = 1.92 avg-val = 114.00 avg-zval = 27.16 usrkeys = 945196 avg-key = 20.00 avg-zkey = 1.92 avg-val = 121.00 avg-zval = 27.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 945196 bytes = 3190048 } seq expand size = 6616372 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0257 GB dict = 1.07 MB all = 0.0317 GB } UnZip/Zip{ index = 10.4428 value = 4.1979 dict = 1.00 all = 4.2279 } Zip/UnZip{ index = 0.0958 value = 0.2382 dict = 1.00 all = 0.2365 } ---------------------------- total value len = 2251.669254 GB avg = 0.200 KB (by entry num) total key len = 306.961593 GB avg = 0.027 KB total ukey len = 237.410892 GB avg = 0.015 KB total ukey num = 15.741469385 Billion total entry num = 11.282783333 Billion write speed all = 38.344901711 MB/sec (with seq num) write speed all = 36.992186938 MB/sec (without seq num) 2024/11/24-03:27:19.013025 139957424981568 [compaction_job.cc:1948] [default] [JOB 40789] Generated table #290306: 945196 keys, 31745840 bytes, temperature: kWarm 2024/11/24-03:27:19.013125 139957424981568 EVENT_LOG_v1 {"time_micros": 1732390039013077, "cf_name": "default", "job": 40789, "event": "table_file_creation", "file_number": 290306, "file_size": 31745840, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4311514916, "largest_seqno": 4413483168, "table_properties": {"data_size": 25667904, "index_size": 1810240, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26465488, "raw_average_key_size": 28, "raw_value_size": 107752344, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 945196, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390035, "oldest_key_time": 0, "file_creation_time": 1732390035, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290306, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:27:19.093616 139957424981568 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290307.sst) /dev/shm/ToplingTemp/Topling-258257-oCFDRo 2024/11/24-03:27:20.061003 139957424981568 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290307.sst): first pass time = 0.97's, 138.750'MB/sec 2024/11/24-03:27:21.447689 139957424981568 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290307.sst): old prealloc_size = 147639500, real_size = 31869040 2024/11/24-03:27:21.448406 139957424981568 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290307.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920271 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1085 GB zip-val = 0.0299 GB avg-val = 117.85 avg-zval = 32.48 2024/11/24-03:27:21.449264 139957424981568 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290307.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:27:22.326702 139957424981568 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290307.sst): second pass time = 1.39's, 78.261'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 40.899'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2517.186'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.319'MB/sec write SST data time = 0.16's, 230.199'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 35021.901'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 78.261'MB/sec zip pipeline throughput = 78.261'MB/sec entries = 920271 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.85 avg-zval = 32.48 usrkeys = 920271 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.85 avg-zval = 32.48 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 920271 bytes = 3220960 } seq expand size = 6441897 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.5146 value = 3.6285 dict = 1.00 all = 3.7335 } Zip/UnZip{ index = 0.0951 value = 0.2756 dict = 1.00 all = 0.2678 } ---------------------------- total value len = 2251.784146 GB avg = 0.200 KB (by entry num) total key len = 306.987360 GB avg = 0.027 KB total ukey len = 237.429297 GB avg = 0.015 KB total ukey num = 15.742389656 Billion total entry num = 11.283703604 Billion write speed all = 38.345080320 MB/sec (with seq num) write speed all = 36.992323280 MB/sec (without seq num) 2024/11/24-03:27:22.372554 139957424981568 [compaction_job.cc:1948] [default] [JOB 40789] Generated table #290307: 920271 keys, 35950016 bytes, temperature: kWarm 2024/11/24-03:27:22.372623 139957424981568 EVENT_LOG_v1 {"time_micros": 1732390042372588, "cf_name": "default", "job": 40789, "event": "table_file_creation", "file_number": 290307, "file_size": 35950016, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679865, "largest_seqno": 4417289586, "table_properties": {"data_size": 29888352, "index_size": 1750464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25767588, "raw_average_key_size": 28, "raw_value_size": 108450154, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920271, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390039, "oldest_key_time": 0, "file_creation_time": 1732390039, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290307, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:27:22.413813 139957424981568 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290308.sst) /dev/shm/ToplingTemp/Topling-258257-HNCMCn 2024/11/24-03:27:23.421093 139957424981568 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290308.sst): first pass time = 1.01's, 133.252'MB/sec 2024/11/24-03:27:24.846661 139957424981568 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290308.sst): old prealloc_size = 147639500, real_size = 30383280 2024/11/24-03:27:24.847235 139957424981568 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290308.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934824 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0284 GB avg-val = 115.58 avg-zval = 30.35 2024/11/24-03:27:24.848007 139957424981568 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290308.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:27:25.684868 139957424981568 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290308.sst): second pass time = 1.42's, 75.831'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 41.167'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3229.177'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.330'MB/sec write SST data time = 0.13's, 264.967'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31778.125'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 75.831'MB/sec zip pipeline throughput = 75.831'MB/sec entries = 934824 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.58 avg-zval = 30.35 usrkeys = 934824 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.58 avg-zval = 30.35 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934824 bytes = 3271904 } seq expand size = 6543768 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0284 GB dict = 1.08 MB all = 0.0345 GB } UnZip/Zip{ index = 10.5118 value = 3.8082 dict = 1.00 all = 3.8897 } Zip/UnZip{ index = 0.0951 value = 0.2626 dict = 1.00 all = 0.2571 } ---------------------------- total value len = 2251.898732 GB avg = 0.200 KB (by entry num) total key len = 307.013535 GB avg = 0.027 KB total ukey len = 237.447994 GB avg = 0.015 KB total ukey num = 15.743324480 Billion total entry num = 11.284638428 Billion write speed all = 38.345260026 MB/sec (with seq num) write speed all = 36.992458993 MB/sec (without seq num) 2024/11/24-03:27:25.727139 139957424981568 [compaction_job.cc:1948] [default] [JOB 40789] Generated table #290308: 934824 keys, 34505608 bytes, temperature: kWarm 2024/11/24-03:27:25.727211 139957424981568 EVENT_LOG_v1 {"time_micros": 1732390045727171, "cf_name": "default", "job": 40789, "event": "table_file_creation", "file_number": 290308, "file_size": 34505608, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679880, "largest_seqno": 4417292214, "table_properties": {"data_size": 28370784, "index_size": 1778624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26175072, "raw_average_key_size": 28, "raw_value_size": 108042708, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934824, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390042, "oldest_key_time": 0, "file_creation_time": 1732390042, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290308, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:27:25.764284 139957424981568 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290309.sst) /dev/shm/ToplingTemp/Topling-258257-Hvluxo 2024/11/24-03:27:26.722868 139957424981568 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290309.sst): first pass time = 0.96's, 140.021'MB/sec 2024/11/24-03:27:28.168086 139957424981568 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290309.sst): old prealloc_size = 147639500, real_size = 31438512 2024/11/24-03:27:28.168611 139957424981568 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290309.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 32.03 2024/11/24-03:27:28.169352 139957424981568 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290309.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:27:28.985251 139957424981568 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290309.sst): second pass time = 1.44's, 75.127'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.322'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3502.290'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 276.137'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 26461.722'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.127'MB/sec zip pipeline throughput = 75.127'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.03 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4443 value = 3.6837 dict = 1.00 all = 3.7914 } Zip/UnZip{ index = 0.0957 value = 0.2715 dict = 1.00 all = 0.2638 } ---------------------------- total value len = 2252.013645 GB avg = 0.200 KB (by entry num) total key len = 307.039276 GB avg = 0.027 KB total ukey len = 237.466380 GB avg = 0.015 KB total ukey num = 15.744243780 Billion total entry num = 11.285557728 Billion write speed all = 38.345471284 MB/sec (with seq num) write speed all = 36.992626953 MB/sec (without seq num) 2024/11/24-03:27:29.029362 139957424981568 [compaction_job.cc:1948] [default] [JOB 40789] Generated table #290309: 919300 keys, 35400370 bytes, temperature: kWarm 2024/11/24-03:27:29.029429 139957424981568 EVENT_LOG_v1 {"time_micros": 1732390049029393, "cf_name": "default", "job": 40789, "event": "table_file_creation", "file_number": 290309, "file_size": 35400370, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4272355128, "largest_seqno": 4381886414, "table_properties": {"data_size": 29448096, "index_size": 1760384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390045, "oldest_key_time": 0, "file_creation_time": 1732390045, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290309, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:27:29.066384 139957424981568 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290310.sst) /dev/shm/ToplingTemp/Topling-258257-atHado 2024/11/24-03:27:30.055585 139957424981568 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290310.sst): first pass time = 0.99's, 135.688'MB/sec 2024/11/24-03:27:31.469941 139957424981568 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290310.sst): old prealloc_size = 147639500, real_size = 31983680 2024/11/24-03:27:31.470717 139957424981568 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290310.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931583 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0300 GB avg-val = 116.07 avg-zval = 32.18 2024/11/24-03:27:31.471627 139957424981568 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290310.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:27:32.340342 139957424981568 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290310.sst): second pass time = 1.41's, 76.501'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 41.470'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2416.453'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.325'MB/sec write SST data time = 0.15's, 234.464'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35778.242'MB/sec zip my value time = 1.41's, unzip length = 0.108'GB zip my value throughput = 76.501'MB/sec zip pipeline throughput = 76.501'MB/sec entries = 931583 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.07 avg-zval = 32.18 usrkeys = 931583 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.07 avg-zval = 32.18 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931583 bytes = 3260560 } seq expand size = 6521081 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.5147 value = 3.6070 dict = 1.00 all = 3.7184 } Zip/UnZip{ index = 0.0951 value = 0.2772 dict = 1.00 all = 0.2689 } ---------------------------- total value len = 2252.128299 GB avg = 0.200 KB (by entry num) total key len = 307.065360 GB avg = 0.027 KB total ukey len = 237.485011 GB avg = 0.015 KB total ukey num = 15.745175363 Billion total entry num = 11.286489311 Billion write speed all = 38.345652382 MB/sec (with seq num) write speed all = 36.992764392 MB/sec (without seq num) 2024/11/24-03:27:32.384192 139957424981568 [compaction_job.cc:1948] [default] [JOB 40789] Generated table #290310: 931583 keys, 36095238 bytes, temperature: kWarm 2024/11/24-03:27:32.384264 139957424981568 EVENT_LOG_v1 {"time_micros": 1732390052384228, "cf_name": "default", "job": 40789, "event": "table_file_creation", "file_number": 290310, "file_size": 36095238, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679891, "largest_seqno": 4417291367, "table_properties": {"data_size": 29978608, "index_size": 1771968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26084324, "raw_average_key_size": 28, "raw_value_size": 108133430, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931583, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390049, "oldest_key_time": 0, "file_creation_time": 1732390049, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290310, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:27:32.419722 139957424981568 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290311.sst) /dev/shm/ToplingTemp/Topling-258257-qqlYPn 2024/11/24-03:27:33.437640 139957424981568 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290311.sst): first pass time = 1.02's, 131.860'MB/sec 2024/11/24-03:27:34.829471 139957424981568 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290311.sst): old prealloc_size = 147639500, real_size = 31316608 2024/11/24-03:27:34.830039 139957424981568 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290311.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1085 GB zip-val = 0.0293 GB avg-val = 118.00 avg-zval = 31.89 2024/11/24-03:27:34.830803 139957424981568 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290311.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:27:35.639667 139957424981568 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290311.sst): second pass time = 1.39's, 78.006'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.544'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3244.794'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.336'MB/sec write SST data time = 0.13's, 281.086'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 32551.602'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 78.006'MB/sec zip pipeline throughput = 78.006'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 31.89 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 31.89 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.3779 value = 3.7004 dict = 1.00 all = 3.8045 } Zip/UnZip{ index = 0.0964 value = 0.2702 dict = 1.00 all = 0.2628 } ---------------------------- total value len = 2252.243212 GB avg = 0.200 KB (by entry num) total key len = 307.091100 GB avg = 0.027 KB total ukey len = 237.503397 GB avg = 0.015 KB total ukey num = 15.746094663 Billion total entry num = 11.287408611 Billion write speed all = 38.345864207 MB/sec (with seq num) write speed all = 36.992932905 MB/sec (without seq num) 2024/11/24-03:27:35.682663 139957424981568 [compaction_job.cc:1948] [default] [JOB 40789] Generated table #290311: 919300 keys, 35278450 bytes, temperature: kWarm 2024/11/24-03:27:35.683405 139957424981568 EVENT_LOG_v1 {"time_micros": 1732390055682693, "cf_name": "default", "job": 40789, "event": "table_file_creation", "file_number": 290311, "file_size": 35278450, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4287823606, "largest_seqno": 4397377712, "table_properties": {"data_size": 29314928, "index_size": 1771648, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390052, "oldest_key_time": 0, "file_creation_time": 1732390052, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290311, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:27:35.722141 139957424981568 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290312.sst) /dev/shm/ToplingTemp/Topling-258257-ymQZDo 2024/11/24-03:27:36.716234 139957424981568 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290312.sst): first pass time = 0.99's, 135.020'MB/sec 2024/11/24-03:27:38.156438 139957424981568 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290312.sst): old prealloc_size = 147639500, real_size = 32796592 2024/11/24-03:27:38.157023 139957424981568 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290312.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930686 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0308 GB avg-val = 116.21 avg-zval = 33.09 2024/11/24-03:27:38.158846 139957424981568 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290312.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:27:39.013427 139957424981568 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290312.sst): second pass time = 1.44's, 75.157'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.066'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3143.439'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 292.716'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32952.177'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.157'MB/sec zip pipeline throughput = 75.157'MB/sec entries = 930686 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.21 avg-zval = 33.09 usrkeys = 930686 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.21 avg-zval = 33.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930686 bytes = 3257408 } seq expand size = 6514802 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5125 value = 3.5124 dict = 1.00 all = 3.6368 } Zip/UnZip{ index = 0.0951 value = 0.2847 dict = 1.00 all = 0.2750 } ---------------------------- total value len = 2252.357885 GB avg = 0.200 KB (by entry num) total key len = 307.117160 GB avg = 0.027 KB total ukey len = 237.522011 GB avg = 0.015 KB total ukey num = 15.747025349 Billion total entry num = 11.288339297 Billion write speed all = 38.346034447 MB/sec (with seq num) write speed all = 36.993059982 MB/sec (without seq num) 2024/11/24-03:27:39.058862 139957424981568 [compaction_job.cc:1948] [default] [JOB 40789] Generated table #290312: 930686 keys, 36905576 bytes, temperature: kWarm 2024/11/24-03:27:39.058932 139957424981568 EVENT_LOG_v1 {"time_micros": 1732390059058896, "cf_name": "default", "job": 40789, "event": "table_file_creation", "file_number": 290312, "file_size": 36905576, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679902, "largest_seqno": 4417292016, "table_properties": {"data_size": 30793120, "index_size": 1770624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26059208, "raw_average_key_size": 28, "raw_value_size": 108158536, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930686, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390055, "oldest_key_time": 0, "file_creation_time": 1732390055, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290312, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:27:39.097218 139957424981568 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290313.sst) /dev/shm/ToplingTemp/Topling-258257-VhwoBn 2024/11/24-03:27:40.048076 139957424981568 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290313.sst): first pass time = 0.95's, 141.160'MB/sec 2024/11/24-03:27:41.484399 139957424981568 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290313.sst): old prealloc_size = 147639500, real_size = 31359744 2024/11/24-03:27:41.484918 139957424981568 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290313.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.95 2024/11/24-03:27:41.486757 139957424981568 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290313.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:27:42.301028 139957424981568 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290313.sst): second pass time = 1.44's, 75.563'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.256'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3551.259'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.13's, 278.696'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 32768.868'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.563'MB/sec zip pipeline throughput = 75.563'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.95 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4455 value = 3.6935 dict = 1.00 all = 3.7999 } Zip/UnZip{ index = 0.0957 value = 0.2707 dict = 1.00 all = 0.2632 } ---------------------------- total value len = 2252.472798 GB avg = 0.200 KB (by entry num) total key len = 307.142900 GB avg = 0.027 KB total ukey len = 237.540397 GB avg = 0.015 KB total ukey num = 15.747944649 Billion total entry num = 11.289258597 Billion write speed all = 38.346252983 MB/sec (with seq num) write speed all = 36.993234977 MB/sec (without seq num) 2024/11/24-03:27:42.343987 139957424981568 [compaction_job.cc:1948] [default] [JOB 40789] Generated table #290313: 919300 keys, 35321586 bytes, temperature: kWarm 2024/11/24-03:27:42.344055 139957424981568 EVENT_LOG_v1 {"time_micros": 1732390062344018, "cf_name": "default", "job": 40789, "event": "table_file_creation", "file_number": 290313, "file_size": 35321586, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4306147927, "largest_seqno": 4416596071, "table_properties": {"data_size": 29369520, "index_size": 1760192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390059, "oldest_key_time": 0, "file_creation_time": 1732390059, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290313, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:27:42.380893 139957424981568 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290314.sst) /dev/shm/ToplingTemp/Topling-258257-eclCMm 2024/11/24-03:27:43.349677 139957424981568 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290314.sst): first pass time = 0.97's, 138.548'MB/sec 2024/11/24-03:27:44.775279 139957424981568 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290314.sst): old prealloc_size = 147639500, real_size = 32877536 2024/11/24-03:27:44.775797 139957424981568 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290314.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930703 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.21 avg-zval = 33.16 2024/11/24-03:27:44.777636 139957424981568 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290314.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:27:45.637275 139957424981568 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290314.sst): second pass time = 1.42's, 75.911'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.909'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3585.939'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 290.607'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33258.283'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 75.911'MB/sec zip pipeline throughput = 80.225'MB/sec entries = 930703 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.21 avg-zval = 33.16 usrkeys = 930703 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.21 avg-zval = 33.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930703 bytes = 3257472 } seq expand size = 6514921 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.4444 value = 3.5045 dict = 1.00 all = 3.6289 } Zip/UnZip{ index = 0.0957 value = 0.2853 dict = 1.00 all = 0.2756 } ---------------------------- total value len = 2252.587471 GB avg = 0.200 KB (by entry num) total key len = 307.168960 GB avg = 0.027 KB total ukey len = 237.559011 GB avg = 0.015 KB total ukey num = 15.748875352 Billion total entry num = 11.290189300 Billion write speed all = 38.346444721 MB/sec (with seq num) write speed all = 36.993382798 MB/sec (without seq num) 2024/11/24-03:27:45.684778 139957424981568 [compaction_job.cc:1948] [default] [JOB 40789] Generated table #290314: 930703 keys, 36986210 bytes, temperature: kWarm 2024/11/24-03:27:45.684849 139957424981568 EVENT_LOG_v1 {"time_micros": 1732390065684813, "cf_name": "default", "job": 40789, "event": "table_file_creation", "file_number": 290314, "file_size": 36986210, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679921, "largest_seqno": 4417291908, "table_properties": {"data_size": 30862480, "index_size": 1782208, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26059684, "raw_average_key_size": 28, "raw_value_size": 108158146, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930703, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390062, "oldest_key_time": 0, "file_creation_time": 1732390062, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290314, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:27:45.729091 139957424981568 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290315.sst) /dev/shm/ToplingTemp/Topling-258257-KEN2Jm 2024/11/24-03:27:46.852659 139957424981568 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290315.sst): first pass time = 1.12's, 119.461'MB/sec 2024/11/24-03:27:48.147046 139957424981568 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290315.sst): old prealloc_size = 147639500, real_size = 31024608 2024/11/24-03:27:48.147656 139957424981568 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290315.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923557 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0290 GB avg-val = 117.33 avg-zval = 31.43 2024/11/24-03:27:48.149466 139957424981568 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290315.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:27:48.963049 139957424981568 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290315.sst): second pass time = 1.29's, 83.767'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.890'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3075.182'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 272.540'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33613.983'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.767'MB/sec zip pipeline throughput = 83.767'MB/sec entries = 923557 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.33 avg-zval = 31.43 usrkeys = 923557 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.33 avg-zval = 31.43 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923557 bytes = 3117024 } seq expand size = 6464899 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0290 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.4448 value = 3.7333 dict = 1.00 all = 3.8351 } Zip/UnZip{ index = 0.0957 value = 0.2679 dict = 1.00 all = 0.2608 } ---------------------------- total value len = 2252.702294 GB avg = 0.200 KB (by entry num) total key len = 307.194819 GB avg = 0.027 KB total ukey len = 237.577482 GB avg = 0.015 KB total ukey num = 15.749798909 Billion total entry num = 11.291112857 Billion write speed all = 38.346641724 MB/sec (with seq num) write speed all = 36.993536532 MB/sec (without seq num) 2024/11/24-03:27:49.007643 139957424981568 [compaction_job.cc:1948] [default] [JOB 40789] Generated table #290315: 923557 keys, 34997444 bytes, temperature: kWarm 2024/11/24-03:27:49.007716 139957424981568 EVENT_LOG_v1 {"time_micros": 1732390069007677, "cf_name": "default", "job": 40789, "event": "table_file_creation", "file_number": 290315, "file_size": 34997444, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4324454292, "largest_seqno": 4417291271, "table_properties": {"data_size": 29025104, "index_size": 1768448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25859596, "raw_average_key_size": 28, "raw_value_size": 108358186, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923557, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390065, "oldest_key_time": 0, "file_creation_time": 1732390065, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290315, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:27:49.052850 139957424981568 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290316.sst) /dev/shm/ToplingTemp/Topling-258257-vdPj9m 2024/11/24-03:27:50.044595 139957424981568 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290316.sst): first pass time = 0.99's, 135.340'MB/sec 2024/11/24-03:27:51.345139 139957424981568 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290316.sst): old prealloc_size = 147639500, real_size = 32966032 2024/11/24-03:27:51.345679 139957424981568 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290316.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926578 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 116.85 avg-zval = 33.41 2024/11/24-03:27:51.347503 139957424981568 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290316.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:27:52.197553 139957424981568 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290316.sst): second pass time = 1.30's, 83.306'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.482'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3448.244'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 291.530'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 27599.571'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.306'MB/sec zip pipeline throughput = 83.306'MB/sec entries = 926578 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.85 avg-zval = 33.41 usrkeys = 926578 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.85 avg-zval = 33.41 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926578 bytes = 3243040 } seq expand size = 6486046 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.4450 value = 3.4972 dict = 1.00 all = 3.6213 } Zip/UnZip{ index = 0.0957 value = 0.2859 dict = 1.00 all = 0.2761 } ---------------------------- total value len = 2252.817054 GB avg = 0.200 KB (by entry num) total key len = 307.220764 GB avg = 0.027 KB total ukey len = 237.596014 GB avg = 0.015 KB total ukey num = 15.750725487 Billion total entry num = 11.292039435 Billion write speed all = 38.346891446 MB/sec (with seq num) write speed all = 36.993740777 MB/sec (without seq num) 2024/11/24-03:27:52.245049 139957424981568 [compaction_job.cc:1948] [default] [JOB 40789] Generated table #290316: 926578 keys, 37063440 bytes, temperature: kWarm 2024/11/24-03:27:52.245128 139957424981568 EVENT_LOG_v1 {"time_micros": 1732390072245093, "cf_name": "default", "job": 40789, "event": "table_file_creation", "file_number": 290316, "file_size": 37063440, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679941, "largest_seqno": 4417291449, "table_properties": {"data_size": 30960000, "index_size": 1774208, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25944184, "raw_average_key_size": 28, "raw_value_size": 108273600, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926578, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390069, "oldest_key_time": 0, "file_creation_time": 1732390069, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290316, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:27:52.285437 139957424981568 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290317.sst) /dev/shm/ToplingTemp/Topling-258257-BcNC3o 2024/11/24-03:27:53.175031 139957424981568 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290317.sst): first pass time = 0.89's, 150.884'MB/sec 2024/11/24-03:27:54.478124 139957424981568 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290317.sst): old prealloc_size = 147639500, real_size = 30519904 2024/11/24-03:27:54.478714 139957424981568 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290317.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927826 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0285 GB avg-val = 116.66 avg-zval = 30.73 2024/11/24-03:27:54.480577 139957424981568 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290317.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:27:55.295964 139957424981568 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290317.sst): second pass time = 1.30's, 83.114'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.190'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3138.388'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.337'MB/sec write SST data time = 0.13's, 268.432'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31003.416'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.114'MB/sec zip pipeline throughput = 83.114'MB/sec entries = 927826 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.66 avg-zval = 30.73 usrkeys = 927826 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.66 avg-zval = 30.73 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927826 bytes = 3131424 } seq expand size = 6494782 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0285 GB dict = 1.08 MB all = 0.0345 GB } UnZip/Zip{ index = 10.4462 value = 3.7963 dict = 1.00 all = 3.8900 } Zip/UnZip{ index = 0.0957 value = 0.2634 dict = 1.00 all = 0.2571 } ---------------------------- total value len = 2252.931787 GB avg = 0.199 KB (by entry num) total key len = 307.246743 GB avg = 0.027 KB total ukey len = 237.614570 GB avg = 0.015 KB total ukey num = 15.751653313 Billion total entry num = 11.292967261 Billion write speed all = 38.347219442 MB/sec (with seq num) write speed all = 36.994020394 MB/sec (without seq num) 2024/11/24-03:27:55.338753 139957424981568 [compaction_job.cc:1948] [default] [JOB 40789] Generated table #290317: 927826 keys, 34503440 bytes, temperature: kWarm 2024/11/24-03:27:55.338829 139957424981568 EVENT_LOG_v1 {"time_micros": 1732390075338790, "cf_name": "default", "job": 40789, "event": "table_file_creation", "file_number": 290317, "file_size": 34503440, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4342314800, "largest_seqno": 4417291286, "table_properties": {"data_size": 28511376, "index_size": 1776384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25979128, "raw_average_key_size": 28, "raw_value_size": 108238600, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927826, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390072, "oldest_key_time": 0, "file_creation_time": 1732390072, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290317, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:27:55.374028 139957424981568 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290318.sst) /dev/shm/ToplingTemp/Topling-258257-f1ECRl 2024/11/24-03:27:56.379541 139957424981568 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290318.sst): first pass time = 1.01's, 133.487'MB/sec 2024/11/24-03:27:57.688380 139957424981568 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290318.sst): old prealloc_size = 147639500, real_size = 32428896 2024/11/24-03:27:57.688969 139957424981568 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290318.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922392 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0304 GB avg-val = 117.51 avg-zval = 33.00 2024/11/24-03:27:57.690949 139957424981568 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290318.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:27:58.520839 139957424981568 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290318.sst): second pass time = 1.31's, 82.866'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.866'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3042.836'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.328'MB/sec write SST data time = 0.13's, 282.383'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33727.106'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.866'MB/sec zip pipeline throughput = 82.866'MB/sec entries = 922392 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.51 avg-zval = 33.00 usrkeys = 922392 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.51 avg-zval = 33.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922392 bytes = 3228384 } seq expand size = 6456744 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5131 value = 3.5604 dict = 1.00 all = 3.6756 } Zip/UnZip{ index = 0.0951 value = 0.2809 dict = 1.00 all = 0.2721 } ---------------------------- total value len = 2253.046635 GB avg = 0.199 KB (by entry num) total key len = 307.272570 GB avg = 0.027 KB total ukey len = 237.633018 GB avg = 0.015 KB total ukey num = 15.752575705 Billion total entry num = 11.293889653 Billion write speed all = 38.347474205 MB/sec (with seq num) write speed all = 36.994229996 MB/sec (without seq num) 2024/11/24-03:27:58.566034 139957424981568 [compaction_job.cc:1948] [default] [JOB 40789] Generated table #290318: 922392 keys, 36515610 bytes, temperature: kWarm 2024/11/24-03:27:58.566109 139957424981568 EVENT_LOG_v1 {"time_micros": 1732390078566068, "cf_name": "default", "job": 40789, "event": "table_file_creation", "file_number": 290318, "file_size": 36515610, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679961, "largest_seqno": 4417290311, "table_properties": {"data_size": 30443344, "index_size": 1754752, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25826976, "raw_average_key_size": 28, "raw_value_size": 108390868, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922392, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390075, "oldest_key_time": 0, "file_creation_time": 1732390075, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290318, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:27:58.603266 139957424981568 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290319.sst) /dev/shm/ToplingTemp/Topling-258257-QsCDyn 2024/11/24-03:27:59.523047 139957424981568 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290319.sst): first pass time = 0.92's, 145.930'MB/sec 2024/11/24-03:28:00.810278 139957424981568 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290319.sst): old prealloc_size = 147639500, real_size = 30586176 2024/11/24-03:28:00.810844 139957424981568 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290319.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930891 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0286 GB avg-val = 116.18 avg-zval = 30.70 2024/11/24-03:28:00.811621 139957424981568 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290319.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:28:01.631077 139957424981568 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290319.sst): second pass time = 1.29's, 84.078'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.273'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3261.643'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 274.232'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33155.833'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.078'MB/sec zip pipeline throughput = 84.078'MB/sec entries = 930891 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.18 avg-zval = 30.70 usrkeys = 930891 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.18 avg-zval = 30.70 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930891 bytes = 3258128 } seq expand size = 6516237 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0286 GB dict = 1.08 MB all = 0.0347 GB } UnZip/Zip{ index = 10.5122 value = 3.7839 dict = 1.00 all = 3.8684 } Zip/UnZip{ index = 0.0951 value = 0.2643 dict = 1.00 all = 0.2585 } ---------------------------- total value len = 2253.161304 GB avg = 0.199 KB (by entry num) total key len = 307.298635 GB avg = 0.027 KB total ukey len = 237.651636 GB avg = 0.015 KB total ukey num = 15.753506596 Billion total entry num = 11.294820544 Billion write speed all = 38.347795679 MB/sec (with seq num) write speed all = 36.994502971 MB/sec (without seq num) 2024/11/24-03:28:01.675264 139957424981568 [compaction_job.cc:1948] [default] [JOB 40789] Generated table #290319: 930891 keys, 34695718 bytes, temperature: kWarm 2024/11/24-03:28:01.675351 139957424981568 EVENT_LOG_v1 {"time_micros": 1732390081675301, "cf_name": "default", "job": 40789, "event": "table_file_creation", "file_number": 290319, "file_size": 34695718, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679976, "largest_seqno": 4417291730, "table_properties": {"data_size": 28582192, "index_size": 1771072, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26064948, "raw_average_key_size": 28, "raw_value_size": 108152874, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930891, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390078, "oldest_key_time": 0, "file_creation_time": 1732390078, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290319, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:28:01.716059 139957424981568 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290320.sst) /dev/shm/ToplingTemp/Topling-258257-aQKjjp 2024/11/24-03:28:02.667679 139957424981568 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290320.sst): first pass time = 0.95's, 141.049'MB/sec 2024/11/24-03:28:03.962231 139957424981568 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290320.sst): old prealloc_size = 147639500, real_size = 31356544 2024/11/24-03:28:03.962793 139957424981568 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290320.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.93 2024/11/24-03:28:03.964789 139957424981568 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290320.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:28:04.775992 139957424981568 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290320.sst): second pass time = 1.29's, 83.886'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.404'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3280.611'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 276.972'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 35658.275'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.886'MB/sec zip pipeline throughput = 83.886'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 31.93 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 31.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.3787 value = 3.6954 dict = 1.00 all = 3.8002 } Zip/UnZip{ index = 0.0964 value = 0.2706 dict = 1.00 all = 0.2631 } ---------------------------- total value len = 2253.276216 GB avg = 0.199 KB (by entry num) total key len = 307.324375 GB avg = 0.027 KB total ukey len = 237.670022 GB avg = 0.015 KB total ukey num = 15.754425896 Billion total entry num = 11.295739844 Billion write speed all = 38.348095991 MB/sec (with seq num) write speed all = 36.994756880 MB/sec (without seq num) 2024/11/24-03:28:04.819679 139957424981568 [compaction_job.cc:1948] [default] [JOB 40789] Generated table #290320: 919300 keys, 35318386 bytes, temperature: kWarm 2024/11/24-03:28:04.819749 139957424981568 EVENT_LOG_v1 {"time_micros": 1732390084819714, "cf_name": "default", "job": 40789, "event": "table_file_creation", "file_number": 290320, "file_size": 35318386, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4270323076, "largest_seqno": 4378368555, "table_properties": {"data_size": 29354992, "index_size": 1771520, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390081, "oldest_key_time": 0, "file_creation_time": 1732390081, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290320, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:28:04.855246 139957424981568 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290321.sst) /dev/shm/ToplingTemp/Topling-258257-nYc1Eo 2024/11/24-03:28:05.417362 139957424981568 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290321.sst): first pass time = 0.56's, 184.689'MB/sec 2024/11/24-03:28:06.427919 139957424981568 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290321.sst): old prealloc_size = 147639500, real_size = 24026992 2024/11/24-03:28:06.428379 139957424981568 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290321.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 722099 prefix = 4 raw-key = 0.0116 GB zip-key = 0.0014 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0836 GB zip-val = 0.0225 GB avg-val = 115.76 avg-zval = 31.11 2024/11/24-03:28:06.428996 139957424981568 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290321.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0018 GB, waited 0.000 sec, Key+Value = 0.104 GB 2024/11/24-03:28:07.091671 139957424981568 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00/290321.sst): second pass time = 1.01's, 82.780'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.66's, 40.733'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3152.536'MB/sec (index lex order gen) rebuild zvType time = 0.56's, 0.320'MB/sec write SST data time = 0.10's, 270.674'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.839'MB, throughput = 20875.874'MB/sec zip my value time = 1.01's, unzip length = 0.084'GB zip my value throughput = 82.780'MB/sec zip pipeline throughput = 82.780'MB/sec entries = 722099 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.76 avg-zval = 31.11 usrkeys = 722099 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.76 avg-zval = 31.11 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 722099 bytes = 2346832 } seq expand size = 5054693 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0144 GB value = 0.0836 GB dict = 0.84 MB all = 0.1038 GB } __ZipSize{ index = 0.0014 GB value = 0.0225 GB dict = 0.84 MB all = 0.0270 GB } UnZip/Zip{ index = 10.4408 value = 3.7213 dict = 1.00 all = 3.8398 } Zip/UnZip{ index = 0.0958 value = 0.2687 dict = 1.00 all = 0.2604 } ---------------------------- total value len = 2253.364862 GB avg = 0.199 KB (by entry num) total key len = 307.344594 GB avg = 0.027 KB total ukey len = 237.684464 GB avg = 0.015 KB total ukey num = 15.755147995 Billion total entry num = 11.296461943 Billion write speed all = 38.348396434 MB/sec (with seq num) write speed all = 36.995017745 MB/sec (without seq num) 2024/11/24-03:28:07.126274 139957424981568 [compaction_job.cc:1948] [default] [JOB 40789] Generated table #290321: 722099 keys, 27035418 bytes, temperature: kWarm 2024/11/24-03:28:07.126346 139957424981568 EVENT_LOG_v1 {"time_micros": 1732390087126307, "cf_name": "default", "job": 40789, "event": "table_file_creation", "file_number": 290321, "file_size": 27035418, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4372659432, "largest_seqno": 4417291318, "table_properties": {"data_size": 22463072, "index_size": 1383232, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 20218772, "raw_average_key_size": 28, "raw_value_size": 83591206, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 722099, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390084, "oldest_key_time": 0, "file_creation_time": 1732390084, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290321, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:28:07.149025 139957424981568 [compaction_job.cc:703] job-40789: subcompact[0], size: 0.627060 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40789/att-00 [290304,290305,290306,290307,290308,290309,290310,290311,290312,290313,290314,290315,290316,290317,290318,290319,290320,290321] 2024/11/24-03:28:07.254373 139957424981568 (Original Log Time 2024/11/24-03:27:03.327546) EVENT_LOG_v1 {"time_micros": 1732390023327503, "job": 40789, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [290300], "files_L3": [290264, 290265, 290266, 290267, 290268, 290269, 290270, 290271, 290272, 290273, 290274, 290275, 290234, 290235, 290236, 290237, 290238, 290239, 290240], "score": 0.932052, "input_data_size": 656258474, "oldest_snapshot_seqno": 4417306353} 2024/11/24-03:28:07.254376 139957424981568 (Original Log Time 2024/11/24-03:28:07.252816) [compaction_job.cc:2022] [default] [JOB 40789] Compacted 1@2 + 19@3 files to L3 => 627059586 bytes 2024/11/24-03:28:07.254377 139957424981568 (Original Log Time 2024/11/24-03:28:07.254302) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 18 0 0 0] max score 0.02, MB/sec: 10.3 rd, 9.8 wr, level 3, files in(1, 19) out(18 +0 blob) MB in(27.1, 598.8 +0.0 blob) out(598.0 +0.0 blob), read-write-amplify(45.2) write-amplify(22.1) OK, records in: 16591340, records dropped: 104508 output_compression: Snappy 2024/11/24-03:28:07.254380 139957424981568 (Original Log Time 2024/11/24-03:28:07.254348) EVENT_LOG_v1 {"time_micros": 1732390087254326, "cf": "default", "job": 40789, "event": "compaction_finished", "compaction_time_micros": 63821489, "compaction_time_cpu_micros": 25720491, "output_level": 3, "num_output_files": 18, "total_output_size": 627059586, "num_input_records": 16591340, "num_output_records": 16486832, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 440319312, "file_range_sync_nanos": 0, "file_fsync_nanos": 12868, "file_prepare_write_nanos": 47329, "lsm_state": [0, 0, 0, 18, 0, 0, 0]} 2024/11/24-03:28:07.380227 139957424981568 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732390023,"executesMs":64059,"compactionJobId":40789,"attempt":0,"compactionInputRawBytes":2400352420,"compactionInputZipBytes":656258474,"compactionOutputRawBytes":0,"compactionOutputZipBytes":627059586,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-db3e284ea82d4313311cf7f5eeb30059-50d037019b9c018a-00"}}, response = 2024/11/24-03:28:07.380293 139957424981568 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-40789/att-00: olev 3, work 64.060 s, result 7.881 ms, install 1.581 ms, input{raw 2.236 GiB zip 625.857 MiB}