2024/11/23-04:47:52.805388 140024944858688 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00: fork to child time = 0.009093 sec 2024/11/23-04:47:52.806198 140024944858688 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00 2024/11/23-04:47:52.813103 140024944858688 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00 2024/11/23-04:47:52.814209 140024944858688 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-04:47:52.814486 140024944858688 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-04:47:52.869147 140024944858688 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-19588/att-00/MANIFEST-206347 2024/11/23-04:47:52.884178 140024944858688 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-19588/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 245339, last_sequence is 3794152820, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 245195 2024/11/23-04:47:52.884183 140024944858688 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-04:47:52.884567 140024944858688 [version_set.cc:5721] Creating manifest 245339 2024/11/23-04:47:52.917642 140024944858688 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-04:47:52.938388 140024944858688 [compaction_job.cc:2437] [default] [JOB 19588] Compacting 1@2 + 17@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-04:47:52.938403 140024944858688 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 14640 Base level 2, inputs: [245273(27MB)], [245323(35MB) 245324(15MB) 245215(35MB) 245216(35MB) 245217(33MB) 245218(34MB) 245219(35MB) 245220(35MB) 245221(35MB) 245222(36MB) 245223(35MB) 245224(35MB) 245225(35MB) 245226(36MB) 245227(35MB) 245242(35MB) 245333(36MB)] 2024/11/23-04:47:52.945094 140024944858688 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-04:47:52.945123 140024944858688 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-04:47:52.945141 140024944858688 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-04:47:52.945146 140024944858688 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-04:47:52.945156 140024944858688 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-04:47:53.399043 140024944858688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245340.sst) /dev/shm/ToplingTemp/Topling-412846-b4PrgH 2024/11/23-04:47:53.943861 140024944858688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245340.sst): first pass time = 0.54's, 246.374'MB/sec 2024/11/23-04:47:55.606205 140024944858688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245340.sst): old prealloc_size = 147639500, real_size = 32809392 2024/11/23-04:47:55.606873 140024944858688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245340.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931454 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 116.09 avg-zval = 33.07 2024/11/23-04:47:55.607729 140024944858688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245340.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:47:56.750264 140024944858688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245340.sst): second pass time = 1.66's, 65.083'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.14's, 32.373'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2756.486'MB/sec (index lex order gen) rebuild zvType time = 0.99's, 0.236'MB/sec write SST data time = 0.16's, 236.625'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 27236.887'MB/sec zip my value time = 1.66's, unzip length = 0.108'GB zip my value throughput = 65.083'MB/sec zip pipeline throughput = 65.083'MB/sec entries = 931454 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.09 avg-zval = 33.07 usrkeys = 931454 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.09 avg-zval = 33.07 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931454 bytes = 3376528 } seq expand size = 6520178 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5147 value = 3.5104 dict = 1.00 all = 3.6239 } Zip/UnZip{ index = 0.0951 value = 0.2849 dict = 1.00 all = 0.2759 } ---------------------------- total value len = 0.114657 GB avg = 0.123 KB (by entry num) total key len = 0.026081 GB avg = 0.028 KB total ukey len = 0.018629 GB avg = 0.020 KB total ukey num = 0.000931454 Billion total entry num = 0.000931454 Billion write speed all = 0.001961952 MB/sec (with seq num) write speed all = 0.001858073 MB/sec (without seq num) 2024/11/23-04:47:56.784616 140024944858688 [compaction_job.cc:1948] [default] [JOB 19588] Generated table #245340: 931454 keys, 37036964 bytes, temperature: kWarm 2024/11/23-04:47:56.784698 140024944858688 EVENT_LOG_v1 {"time_micros": 1732308476784647, "cf_name": "default", "job": 19588, "event": "table_file_creation", "file_number": 245340, "file_size": 37036964, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3504428200, "largest_seqno": 3793637746, "table_properties": {"data_size": 30804640, "index_size": 1771712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26080712, "raw_average_key_size": 28, "raw_value_size": 108137116, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931454, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732308473, "oldest_key_time": 0, "file_creation_time": 1732308473, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 245340, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:47:56.824270 140024944858688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245341.sst) /dev/shm/ToplingTemp/Topling-412846-lgWYTG 2024/11/23-04:47:57.952461 140024944858688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245341.sst): first pass time = 1.13's, 118.972'MB/sec 2024/11/23-04:47:59.392982 140024944858688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245341.sst): old prealloc_size = 147639500, real_size = 32821056 2024/11/23-04:47:59.393574 140024944858688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245341.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932061 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 116.00 avg-zval = 33.06 2024/11/23-04:47:59.394303 140024944858688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245341.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:48:00.298157 140024944858688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245341.sst): second pass time = 1.43's, 75.495'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 40.442'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3117.445'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.302'MB/sec write SST data time = 0.13's, 276.174'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33738.150'MB/sec zip my value time = 1.43's, unzip length = 0.108'GB zip my value throughput = 75.495'MB/sec zip pipeline throughput = 75.495'MB/sec entries = 932061 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.00 avg-zval = 33.06 usrkeys = 932061 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.98 avg-zval = 33.06 TagRS{ kinds = RS_Key0_TagN bytes = 145720 } TagArray{ size = 795844 bytes = 2785472 } seq expand size = 5570908 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5129 value = 3.5087 dict = 1.00 all = 3.6665 } Zip/UnZip{ index = 0.0951 value = 0.2850 dict = 1.00 all = 0.2727 } ---------------------------- total value len = 0.228348 GB avg = 0.123 KB (by entry num) total key len = 0.052178 GB avg = 0.028 KB total ukey len = 0.037270 GB avg = 0.020 KB total ukey num = 0.001863515 Billion total entry num = 0.001863515 Billion write speed all = 0.003910478 MB/sec (with seq num) write speed all = 0.003702662 MB/sec (without seq num) 2024/11/23-04:48:00.332439 140024944858688 [compaction_job.cc:1948] [default] [JOB 19588] Generated table #245341: 932061 keys, 36606704 bytes, temperature: kWarm 2024/11/23-04:48:00.332504 140024944858688 EVENT_LOG_v1 {"time_micros": 1732308480332468, "cf_name": "default", "job": 19588, "event": "table_file_creation", "file_number": 245341, "file_size": 36606704, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3793637317, "table_properties": {"data_size": 30814704, "index_size": 1773184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26097708, "raw_average_key_size": 28, "raw_value_size": 108120110, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 932061, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732308476, "oldest_key_time": 0, "file_creation_time": 1732308476, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 245341, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:48:00.382547 140024944858688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245342.sst) /dev/shm/ToplingTemp/Topling-412846-uK9yIH 2024/11/23-04:48:01.283713 140024944858688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245342.sst): first pass time = 0.90's, 148.944'MB/sec 2024/11/23-04:48:02.769318 140024944858688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245342.sst): old prealloc_size = 147639500, real_size = 34977984 2024/11/23-04:48:02.769959 140024944858688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245342.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1081470 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1039 GB zip-val = 0.0327 GB avg-val = 96.11 avg-zval = 30.19 2024/11/23-04:48:02.770002 140024944858688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245342.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:48:03.395279 140024944858688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245342.sst): second pass time = 1.48's, 70.011'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.63's, 57.115'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3338.390'MB/sec (index lex order gen) rebuild zvType time = 0.48's, 0.559'MB/sec write SST data time = 0.14's, 253.033'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.032'MB, throughput = 30305.107'MB/sec zip my value time = 1.48's, unzip length = 0.104'GB zip my value throughput = 70.011'MB/sec zip pipeline throughput = 70.011'MB/sec entries = 1081470 avg-key = 28.00 avg-zkey = 1.90 avg-val = 96.11 avg-zval = 30.19 usrkeys = 1081470 avg-key = 20.00 avg-zkey = 1.90 avg-val = 96.11 avg-zval = 30.19 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0216 GB value = 0.1039 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0327 GB dict = 1.03 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5159 value = 3.1833 dict = 1.00 all = 3.7542 } Zip/UnZip{ index = 0.0951 value = 0.3141 dict = 1.00 all = 0.2664 } ---------------------------- total value len = 0.332285 GB avg = 0.113 KB (by entry num) total key len = 0.082460 GB avg = 0.028 KB total ukey len = 0.058900 GB avg = 0.020 KB total ukey num = 0.002944985 Billion total entry num = 0.002944985 Billion write speed all = 0.005781193 MB/sec (with seq num) write speed all = 0.005452788 MB/sec (without seq num) 2024/11/23-04:48:03.428194 140024944858688 [compaction_job.cc:1948] [default] [JOB 19588] Generated table #245342: 1081470 keys, 35751025 bytes, temperature: kWarm 2024/11/23-04:48:03.428277 140024944858688 EVENT_LOG_v1 {"time_micros": 1732308483428228, "cf_name": "default", "job": 19588, "event": "table_file_creation", "file_number": 245342, "file_size": 35751025, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 32650608, "index_size": 2056832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30281160, "raw_average_key_size": 28, "raw_value_size": 103936657, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1081470, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732308480, "oldest_key_time": 0, "file_creation_time": 1732308480, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 245342, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:48:03.465587 140024944858688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245343.sst) /dev/shm/ToplingTemp/Topling-412846-GcgGDG 2024/11/23-04:48:04.325298 140024944858688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245343.sst): first pass time = 0.86's, 156.125'MB/sec 2024/11/23-04:48:05.668910 140024944858688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245343.sst): old prealloc_size = 147639500, real_size = 33256416 2024/11/23-04:48:05.669500 140024944858688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245343.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920773 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0313 GB avg-val = 117.77 avg-zval = 33.97 2024/11/23-04:48:05.670220 140024944858688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245343.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:48:06.547851 140024944858688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245343.sst): second pass time = 1.34's, 80.751'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 42.077'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3094.869'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.309'MB/sec write SST data time = 0.13's, 275.051'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 32581.906'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.751'MB/sec zip pipeline throughput = 80.751'MB/sec entries = 920773 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.77 avg-zval = 33.97 usrkeys = 920773 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.47 avg-zval = 33.97 TagRS{ kinds = RS_Key0_TagN bytes = 143960 } TagArray{ size = 750057 bytes = 2718976 } seq expand size = 5250399 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5119 value = 3.4673 dict = 1.00 all = 3.6292 } Zip/UnZip{ index = 0.0951 value = 0.2884 dict = 1.00 all = 0.2755 } ---------------------------- total value len = 0.445971 GB avg = 0.115 KB (by entry num) total key len = 0.108241 GB avg = 0.028 KB total ukey len = 0.077315 GB avg = 0.020 KB total ukey num = 0.003865758 Billion total entry num = 0.003865758 Billion write speed all = 0.007724924 MB/sec (with seq num) write speed all = 0.007293859 MB/sec (without seq num) 2024/11/23-04:48:06.584357 140024944858688 [compaction_job.cc:1948] [default] [JOB 19588] Generated table #245343: 920773 keys, 36982704 bytes, temperature: kWarm 2024/11/23-04:48:06.584425 140024944858688 EVENT_LOG_v1 {"time_micros": 1732308486584391, "cf_name": "default", "job": 19588, "event": "table_file_creation", "file_number": 245343, "file_size": 36982704, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3754922846, "table_properties": {"data_size": 31274192, "index_size": 1751872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25781644, "raw_average_key_size": 28, "raw_value_size": 108436086, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920773, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732308483, "oldest_key_time": 0, "file_creation_time": 1732308483, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 245343, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:48:06.622307 140024944858688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245344.sst) /dev/shm/ToplingTemp/Topling-412846-V5SDXE 2024/11/23-04:48:07.516953 140024944858688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245344.sst): first pass time = 0.89's, 150.030'MB/sec 2024/11/23-04:48:08.852415 140024944858688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245344.sst): old prealloc_size = 147639500, real_size = 33362096 2024/11/23-04:48:08.852976 140024944858688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245344.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927771 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0314 GB avg-val = 116.67 avg-zval = 33.81 2024/11/23-04:48:08.853772 140024944858688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245344.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:48:09.804000 140024944858688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245344.sst): second pass time = 1.33's, 81.096'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.492'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3315.690'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.283'MB/sec write SST data time = 0.13's, 286.457'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32398.717'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.096'MB/sec zip pipeline throughput = 81.096'MB/sec entries = 927771 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.67 avg-zval = 33.81 usrkeys = 927771 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.67 avg-zval = 33.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927771 bytes = 3363184 } seq expand size = 6494397 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5119 value = 3.4510 dict = 1.00 all = 3.5716 } Zip/UnZip{ index = 0.0951 value = 0.2898 dict = 1.00 all = 0.2800 } ---------------------------- total value len = 0.560706 GB avg = 0.117 KB (by entry num) total key len = 0.134219 GB avg = 0.028 KB total ukey len = 0.095871 GB avg = 0.020 KB total ukey num = 0.004793529 Billion total entry num = 0.004793529 Billion write speed all = 0.009685808 MB/sec (with seq num) write speed all = 0.009151313 MB/sec (without seq num) 2024/11/23-04:48:09.839340 140024944858688 [compaction_job.cc:1948] [default] [JOB 19588] Generated table #245344: 927771 keys, 37579552 bytes, temperature: kWarm 2024/11/23-04:48:09.839404 140024944858688 EVENT_LOG_v1 {"time_micros": 1732308489839371, "cf_name": "default", "job": 19588, "event": "table_file_creation", "file_number": 245344, "file_size": 37579552, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3487497056, "largest_seqno": 3793636682, "table_properties": {"data_size": 31364768, "index_size": 1765184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25977588, "raw_average_key_size": 28, "raw_value_size": 108240174, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927771, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732308486, "oldest_key_time": 0, "file_creation_time": 1732308486, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 245344, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:48:09.878847 140024944858688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245345.sst) /dev/shm/ToplingTemp/Topling-412846-35lGEG 2024/11/23-04:48:10.759837 140024944858688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245345.sst): first pass time = 0.88's, 152.355'MB/sec 2024/11/23-04:48:12.078685 140024944858688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245345.sst): old prealloc_size = 147639500, real_size = 31846560 2024/11/23-04:48:12.079268 140024944858688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245345.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927840 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0298 GB avg-val = 116.66 avg-zval = 32.16 2024/11/23-04:48:12.080054 140024944858688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245345.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:48:12.970540 140024944858688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245345.sst): second pass time = 1.32's, 82.134'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 40.438'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3163.020'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.305'MB/sec write SST data time = 0.13's, 274.596'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30203.926'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.134'MB/sec zip pipeline throughput = 82.134'MB/sec entries = 927840 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.66 avg-zval = 32.16 usrkeys = 927840 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.66 avg-zval = 32.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927840 bytes = 3363440 } seq expand size = 6494880 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0298 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.4434 value = 3.6276 dict = 1.00 all = 3.7217 } Zip/UnZip{ index = 0.0958 value = 0.2757 dict = 1.00 all = 0.2687 } ---------------------------- total value len = 0.675439 GB avg = 0.118 KB (by entry num) total key len = 0.160198 GB avg = 0.028 KB total ukey len = 0.114427 GB avg = 0.020 KB total ukey num = 0.005721369 Billion total entry num = 0.005721369 Billion write speed all = 0.011646536 MB/sec (with seq num) write speed all = 0.011008612 MB/sec (without seq num) 2024/11/23-04:48:13.010561 140024944858688 [compaction_job.cc:1948] [default] [JOB 19588] Generated table #245345: 927840 keys, 36063856 bytes, temperature: kWarm 2024/11/23-04:48:13.010623 140024944858688 EVENT_LOG_v1 {"time_micros": 1732308493010591, "cf_name": "default", "job": 19588, "event": "table_file_creation", "file_number": 245345, "file_size": 36063856, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3375334738, "largest_seqno": 3793636692, "table_properties": {"data_size": 29837520, "index_size": 1776896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25979520, "raw_average_key_size": 28, "raw_value_size": 108238232, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927840, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732308489, "oldest_key_time": 0, "file_creation_time": 1732308489, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 245345, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:48:13.051823 140024944858688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245346.sst) /dev/shm/ToplingTemp/Topling-412846-BOXAnG 2024/11/23-04:48:13.854225 140024944858688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245346.sst): first pass time = 0.80's, 167.277'MB/sec 2024/11/23-04:48:15.196899 140024944858688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245346.sst): old prealloc_size = 147639500, real_size = 32729840 2024/11/23-04:48:15.197448 140024944858688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245346.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927844 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0307 GB avg-val = 116.66 avg-zval = 33.12 2024/11/23-04:48:15.198252 140024944858688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245346.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:48:16.107934 140024944858688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245346.sst): second pass time = 1.34's, 80.679'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 40.556'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3365.923'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.298'MB/sec write SST data time = 0.13's, 281.746'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30853.748'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.679'MB/sec zip pipeline throughput = 80.679'MB/sec entries = 927844 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.66 avg-zval = 33.12 usrkeys = 927844 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.66 avg-zval = 33.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927844 bytes = 3363456 } seq expand size = 6494908 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5112 value = 3.5220 dict = 1.00 all = 3.6327 } Zip/UnZip{ index = 0.0951 value = 0.2839 dict = 1.00 all = 0.2753 } ---------------------------- total value len = 0.790172 GB avg = 0.119 KB (by entry num) total key len = 0.186178 GB avg = 0.028 KB total ukey len = 0.132984 GB avg = 0.020 KB total ukey num = 0.006649213 Billion total entry num = 0.006649213 Billion write speed all = 0.013607096 MB/sec (with seq num) write speed all = 0.012865751 MB/sec (without seq num) 2024/11/23-04:48:16.141202 140024944858688 [compaction_job.cc:1948] [default] [JOB 19588] Generated table #245346: 927844 keys, 36947344 bytes, temperature: kWarm 2024/11/23-04:48:16.141264 140024944858688 EVENT_LOG_v1 {"time_micros": 1732308496141232, "cf_name": "default", "job": 19588, "event": "table_file_creation", "file_number": 245346, "file_size": 36947344, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3793636709, "table_properties": {"data_size": 30732256, "index_size": 1765440, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25979632, "raw_average_key_size": 28, "raw_value_size": 108238100, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927844, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732308493, "oldest_key_time": 0, "file_creation_time": 1732308493, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 245346, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:48:16.175359 140024944858688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245347.sst) /dev/shm/ToplingTemp/Topling-412846-WVhU1H 2024/11/23-04:48:16.985239 140024944858688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245347.sst): first pass time = 0.81's, 165.732'MB/sec 2024/11/23-04:48:18.300594 140024944858688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245347.sst): old prealloc_size = 147639500, real_size = 30631216 2024/11/23-04:48:18.301216 140024944858688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245347.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927990 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0286 GB avg-val = 116.63 avg-zval = 30.83 2024/11/23-04:48:18.301975 140024944858688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245347.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:48:19.159398 140024944858688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245347.sst): second pass time = 1.31's, 82.359'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 40.442'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3011.975'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 269.068'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31918.307'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.359'MB/sec zip pipeline throughput = 82.359'MB/sec entries = 927990 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.63 avg-zval = 30.83 usrkeys = 927990 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.63 avg-zval = 30.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927990 bytes = 3247984 } seq expand size = 6495930 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0286 GB dict = 1.08 MB all = 0.0347 GB } UnZip/Zip{ index = 10.3770 value = 3.7830 dict = 1.00 all = 3.8645 } Zip/UnZip{ index = 0.0964 value = 0.2643 dict = 1.00 all = 0.2588 } ---------------------------- total value len = 0.904902 GB avg = 0.119 KB (by entry num) total key len = 0.212162 GB avg = 0.028 KB total ukey len = 0.151544 GB avg = 0.020 KB total ukey num = 0.007577203 Billion total entry num = 0.007577203 Billion write speed all = 0.015567518 MB/sec (with seq num) write speed all = 0.014722745 MB/sec (without seq num) 2024/11/23-04:48:19.194433 140024944858688 [compaction_job.cc:1948] [default] [JOB 19588] Generated table #245347: 927990 keys, 34731076 bytes, temperature: kWarm 2024/11/23-04:48:19.194498 140024944858688 EVENT_LOG_v1 {"time_micros": 1732308499194465, "cf_name": "default", "job": 19588, "event": "table_file_creation", "file_number": 245347, "file_size": 34731076, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3636599133, "largest_seqno": 3793636723, "table_properties": {"data_size": 28610464, "index_size": 1788544, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25983720, "raw_average_key_size": 28, "raw_value_size": 108234052, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927990, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732308496, "oldest_key_time": 0, "file_creation_time": 1732308496, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 245347, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:48:19.231009 140024944858688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245348.sst) /dev/shm/ToplingTemp/Topling-412846-mIG1pI 2024/11/23-04:48:20.137839 140024944858688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245348.sst): first pass time = 0.91's, 148.014'MB/sec 2024/11/23-04:48:21.464544 140024944858688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245348.sst): old prealloc_size = 147639500, real_size = 30857504 2024/11/23-04:48:21.465139 140024944858688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245348.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928166 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0289 GB avg-val = 116.61 avg-zval = 31.09 2024/11/23-04:48:21.466847 140024944858688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245348.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:48:22.324579 140024944858688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245348.sst): second pass time = 1.33's, 81.620'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 40.785'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3097.053'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 266.645'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 29389.645'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.620'MB/sec zip pipeline throughput = 81.620'MB/sec entries = 928166 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.61 avg-zval = 31.09 usrkeys = 928166 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.61 avg-zval = 31.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928166 bytes = 3364624 } seq expand size = 6497162 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.5129 value = 3.7502 dict = 1.00 all = 3.8265 } Zip/UnZip{ index = 0.0951 value = 0.2667 dict = 1.00 all = 0.2613 } ---------------------------- total value len = 1.019628 GB avg = 0.120 KB (by entry num) total key len = 0.238150 GB avg = 0.028 KB total ukey len = 0.170107 GB avg = 0.020 KB total ukey num = 0.008505369 Billion total entry num = 0.008505369 Billion write speed all = 0.017527763 MB/sec (with seq num) write speed all = 0.016579552 MB/sec (without seq num) 2024/11/23-04:48:22.356108 140024944858688 [compaction_job.cc:1948] [default] [JOB 19588] Generated table #245348: 928166 keys, 35075578 bytes, temperature: kWarm 2024/11/23-04:48:22.356175 140024944858688 EVENT_LOG_v1 {"time_micros": 1732308502356143, "cf_name": "default", "job": 19588, "event": "table_file_creation", "file_number": 245348, "file_size": 35075578, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369194706, "largest_seqno": 3793636879, "table_properties": {"data_size": 28859536, "index_size": 1765760, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25988648, "raw_average_key_size": 28, "raw_value_size": 108229156, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928166, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732308499, "oldest_key_time": 0, "file_creation_time": 1732308499, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 245348, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:48:22.395967 140024944858688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245349.sst) /dev/shm/ToplingTemp/Topling-412846-Y8PxkH 2024/11/23-04:48:23.277737 140024944858688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245349.sst): first pass time = 0.88's, 152.220'MB/sec 2024/11/23-04:48:24.597419 140024944858688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245349.sst): old prealloc_size = 147639500, real_size = 33508416 2024/11/23-04:48:24.598010 140024944858688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245349.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925659 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 117.00 avg-zval = 34.05 2024/11/23-04:48:24.598826 140024944858688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245349.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:48:25.535648 140024944858688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245349.sst): second pass time = 1.32's, 82.110'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 40.203'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3091.146'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.287'MB/sec write SST data time = 0.13's, 288.962'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33598.246'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.110'MB/sec zip pipeline throughput = 82.110'MB/sec entries = 925659 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.00 avg-zval = 34.05 usrkeys = 925659 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.00 avg-zval = 34.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925659 bytes = 3355536 } seq expand size = 6479613 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5108 value = 3.4364 dict = 1.00 all = 3.5584 } Zip/UnZip{ index = 0.0951 value = 0.2910 dict = 1.00 all = 0.2810 } ---------------------------- total value len = 1.134407 GB avg = 0.120 KB (by entry num) total key len = 0.264069 GB avg = 0.028 KB total ukey len = 0.188621 GB avg = 0.020 KB total ukey num = 0.009431028 Billion total entry num = 0.009431028 Billion write speed all = 0.019487578 MB/sec (with seq num) write speed all = 0.018436217 MB/sec (without seq num) 2024/11/23-04:48:25.571500 140024944858688 [compaction_job.cc:1948] [default] [JOB 19588] Generated table #245349: 925659 keys, 37718902 bytes, temperature: kWarm 2024/11/23-04:48:25.571564 140024944858688 EVENT_LOG_v1 {"time_micros": 1732308505571531, "cf_name": "default", "job": 19588, "event": "table_file_creation", "file_number": 245349, "file_size": 37718902, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3373729869, "largest_seqno": 3793637634, "table_properties": {"data_size": 31515504, "index_size": 1761344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25918452, "raw_average_key_size": 28, "raw_value_size": 108299298, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925659, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732308502, "oldest_key_time": 0, "file_creation_time": 1732308502, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 245349, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:48:25.606467 140024944858688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245350.sst) /dev/shm/ToplingTemp/Topling-412846-jiVCvH 2024/11/23-04:48:26.475725 140024944858688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245350.sst): first pass time = 0.87's, 154.411'MB/sec 2024/11/23-04:48:27.816710 140024944858688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245350.sst): old prealloc_size = 147639500, real_size = 32500544 2024/11/23-04:48:27.817261 140024944858688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245350.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930967 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0305 GB avg-val = 116.17 avg-zval = 32.76 2024/11/23-04:48:27.818902 140024944858688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245350.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:48:28.717001 140024944858688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245350.sst): second pass time = 1.34's, 80.691'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 40.795'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3352.856'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.303'MB/sec write SST data time = 0.13's, 280.973'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33727.745'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.691'MB/sec zip pipeline throughput = 80.691'MB/sec entries = 930967 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.17 avg-zval = 32.76 usrkeys = 930967 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.17 avg-zval = 32.76 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930967 bytes = 3374768 } seq expand size = 6516769 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5127 value = 3.5463 dict = 1.00 all = 3.6545 } Zip/UnZip{ index = 0.0951 value = 0.2820 dict = 1.00 all = 0.2736 } ---------------------------- total value len = 1.249075 GB avg = 0.121 KB (by entry num) total key len = 0.290136 GB avg = 0.028 KB total ukey len = 0.207240 GB avg = 0.020 KB total ukey num = 0.010361995 Billion total entry num = 0.010361995 Billion write speed all = 0.021447745 MB/sec (with seq num) write speed all = 0.020292652 MB/sec (without seq num) 2024/11/23-04:48:28.752086 140024944858688 [compaction_job.cc:1948] [default] [JOB 19588] Generated table #245350: 930967 keys, 36726676 bytes, temperature: kWarm 2024/11/23-04:48:28.752149 140024944858688 EVENT_LOG_v1 {"time_micros": 1732308508752114, "cf_name": "default", "job": 19588, "event": "table_file_creation", "file_number": 245350, "file_size": 36726676, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 3793637088, "table_properties": {"data_size": 30496496, "index_size": 1771136, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26067076, "raw_average_key_size": 28, "raw_value_size": 108150770, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930967, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732308505, "oldest_key_time": 0, "file_creation_time": 1732308505, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 245350, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:48:28.786493 140024944858688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245351.sst) /dev/shm/ToplingTemp/Topling-412846-TZCHSD 2024/11/23-04:48:29.575210 140024944858688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245351.sst): first pass time = 0.79's, 170.179'MB/sec 2024/11/23-04:48:30.905951 140024944858688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245351.sst): old prealloc_size = 147639500, real_size = 32589312 2024/11/23-04:48:30.906489 140024944858688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245351.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927927 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0306 GB avg-val = 116.64 avg-zval = 32.97 2024/11/23-04:48:30.907331 140024944858688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245351.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:48:31.821965 140024944858688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245351.sst): second pass time = 1.33's, 81.378'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.182'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3437.552'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.295'MB/sec write SST data time = 0.13's, 282.590'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30549.559'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.378'MB/sec zip pipeline throughput = 81.378'MB/sec entries = 927927 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.64 avg-zval = 32.97 usrkeys = 927927 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.64 avg-zval = 32.97 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927927 bytes = 3363744 } seq expand size = 6495489 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5114 value = 3.5381 dict = 1.00 all = 3.6465 } Zip/UnZip{ index = 0.0951 value = 0.2826 dict = 1.00 all = 0.2742 } ---------------------------- total value len = 1.363806 GB avg = 0.121 KB (by entry num) total key len = 0.316118 GB avg = 0.028 KB total ukey len = 0.225798 GB avg = 0.020 KB total ukey num = 0.011289922 Billion total entry num = 0.011289922 Billion write speed all = 0.023407466 MB/sec (with seq num) write speed all = 0.022148988 MB/sec (without seq num) 2024/11/23-04:48:31.854961 140024944858688 [compaction_job.cc:1948] [default] [JOB 19588] Generated table #245351: 927927 keys, 36806916 bytes, temperature: kWarm 2024/11/23-04:48:31.855025 140024944858688 EVENT_LOG_v1 {"time_micros": 1732308511854993, "cf_name": "default", "job": 19588, "event": "table_file_creation", "file_number": 245351, "file_size": 36806916, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 3793636906, "table_properties": {"data_size": 30591600, "index_size": 1765568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25981956, "raw_average_key_size": 28, "raw_value_size": 108235814, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927927, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732308508, "oldest_key_time": 0, "file_creation_time": 1732308508, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 245351, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:48:31.893037 140024944858688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245352.sst) /dev/shm/ToplingTemp/Topling-412846-IzNqzG 2024/11/23-04:48:32.705960 140024944858688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245352.sst): first pass time = 0.81's, 165.113'MB/sec 2024/11/23-04:48:34.049338 140024944858688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245352.sst): old prealloc_size = 147639500, real_size = 33335552 2024/11/23-04:48:34.049891 140024944858688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245352.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929884 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0313 GB avg-val = 116.34 avg-zval = 33.68 2024/11/23-04:48:34.050567 140024944858688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245352.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:48:34.921700 140024944858688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245352.sst): second pass time = 1.34's, 80.568'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.466'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3350.126'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.13's, 278.383'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 23803.308'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.568'MB/sec zip pipeline throughput = 80.568'MB/sec entries = 929884 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.34 avg-zval = 33.68 usrkeys = 929884 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.96 avg-zval = 33.68 TagRS{ kinds = RS_Key0_TagN bytes = 145400 } TagArray{ size = 746887 bytes = 2707488 } seq expand size = 5228209 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.4476 value = 3.4537 dict = 1.00 all = 3.6231 } Zip/UnZip{ index = 0.0957 value = 0.2895 dict = 1.00 all = 0.2760 } ---------------------------- total value len = 1.477215 GB avg = 0.121 KB (by entry num) total key len = 0.342155 GB avg = 0.028 KB total ukey len = 0.244396 GB avg = 0.020 KB total ukey num = 0.012219806 Billion total entry num = 0.012219806 Billion write speed all = 0.025349363 MB/sec (with seq num) write speed all = 0.023987290 MB/sec (without seq num) 2024/11/23-04:48:34.956552 140024944858688 [compaction_job.cc:1948] [default] [JOB 19588] Generated table #245352: 929884 keys, 37044634 bytes, temperature: kWarm 2024/11/23-04:48:34.956615 140024944858688 EVENT_LOG_v1 {"time_micros": 1732308514956583, "cf_name": "default", "job": 19588, "event": "table_file_creation", "file_number": 245352, "file_size": 37044634, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3793637350, "table_properties": {"data_size": 31322800, "index_size": 1780096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26036752, "raw_average_key_size": 28, "raw_value_size": 108181092, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929884, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732308511, "oldest_key_time": 0, "file_creation_time": 1732308511, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 245352, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:48:34.990050 140024944858688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245353.sst) /dev/shm/ToplingTemp/Topling-412846-q7cY5E 2024/11/23-04:48:35.847881 140024944858688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245353.sst): first pass time = 0.86's, 156.468'MB/sec 2024/11/23-04:48:37.184546 140024944858688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245353.sst): old prealloc_size = 147639500, real_size = 32504880 2024/11/23-04:48:37.185150 140024944858688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245353.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928056 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0305 GB avg-val = 116.62 avg-zval = 32.87 2024/11/23-04:48:37.185960 140024944858688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245353.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:48:38.095378 140024944858688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245353.sst): second pass time = 1.34's, 81.058'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 40.319'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3046.783'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.298'MB/sec write SST data time = 0.13's, 277.212'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 21077.509'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.058'MB/sec zip pipeline throughput = 81.058'MB/sec entries = 928056 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.62 avg-zval = 32.87 usrkeys = 928056 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.62 avg-zval = 32.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928056 bytes = 3364224 } seq expand size = 6496392 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5125 value = 3.5478 dict = 1.00 all = 3.6549 } Zip/UnZip{ index = 0.0951 value = 0.2819 dict = 1.00 all = 0.2736 } ---------------------------- total value len = 1.591944 GB avg = 0.121 KB (by entry num) total key len = 0.368140 GB avg = 0.028 KB total ukey len = 0.262957 GB avg = 0.020 KB total ukey num = 0.013147862 Billion total entry num = 0.013147862 Billion write speed all = 0.027308732 MB/sec (with seq num) write speed all = 0.025843279 MB/sec (without seq num) 2024/11/23-04:48:38.129538 140024944858688 [compaction_job.cc:1948] [default] [JOB 19588] Generated table #245353: 928056 keys, 36722824 bytes, temperature: kWarm 2024/11/23-04:48:38.129601 140024944858688 EVENT_LOG_v1 {"time_micros": 1732308518129569, "cf_name": "default", "job": 19588, "event": "table_file_creation", "file_number": 245353, "file_size": 36722824, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3288481706, "largest_seqno": 3793637173, "table_properties": {"data_size": 30507040, "index_size": 1765632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25985568, "raw_average_key_size": 28, "raw_value_size": 108232220, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928056, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732308514, "oldest_key_time": 0, "file_creation_time": 1732308514, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 245353, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:48:38.164429 140024944858688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245354.sst) /dev/shm/ToplingTemp/Topling-412846-jcaQ0E 2024/11/23-04:48:39.069626 140024944858688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245354.sst): first pass time = 0.91's, 148.280'MB/sec 2024/11/23-04:48:40.395850 140024944858688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245354.sst): old prealloc_size = 147639500, real_size = 32180000 2024/11/23-04:48:40.396427 140024944858688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245354.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927890 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0302 GB avg-val = 116.65 avg-zval = 32.52 2024/11/23-04:48:40.399060 140024944858688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245354.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:48:41.302463 140024944858688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245354.sst): second pass time = 1.32's, 81.694'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 40.148'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3201.118'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.300'MB/sec write SST data time = 0.13's, 273.111'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 25540.645'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.694'MB/sec zip pipeline throughput = 81.694'MB/sec entries = 927890 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.65 avg-zval = 32.52 usrkeys = 927890 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.65 avg-zval = 32.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927890 bytes = 3363616 } seq expand size = 6495230 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.4443 value = 3.5874 dict = 1.00 all = 3.6876 } Zip/UnZip{ index = 0.0957 value = 0.2788 dict = 1.00 all = 0.2712 } ---------------------------- total value len = 1.706676 GB avg = 0.121 KB (by entry num) total key len = 0.394121 GB avg = 0.028 KB total ukey len = 0.281515 GB avg = 0.020 KB total ukey num = 0.014075752 Billion total entry num = 0.014075752 Billion write speed all = 0.029267898 MB/sec (with seq num) write speed all = 0.027699093 MB/sec (without seq num) 2024/11/23-04:48:41.344089 140024944858688 [compaction_job.cc:1948] [default] [JOB 19588] Generated table #245354: 927890 keys, 36397468 bytes, temperature: kWarm 2024/11/23-04:48:41.344156 140024944858688 EVENT_LOG_v1 {"time_micros": 1732308521344123, "cf_name": "default", "job": 19588, "event": "table_file_creation", "file_number": 245354, "file_size": 36397468, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 3793637184, "table_properties": {"data_size": 30171024, "index_size": 1776832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25980920, "raw_average_key_size": 28, "raw_value_size": 108236816, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927890, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732308518, "oldest_key_time": 0, "file_creation_time": 1732308518, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 245354, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:48:41.386866 140024944858688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245355.sst) /dev/shm/ToplingTemp/Topling-412846-YbeHSG 2024/11/23-04:48:42.304610 140024944858688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245355.sst): first pass time = 0.92's, 146.253'MB/sec 2024/11/23-04:48:43.639236 140024944858688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245355.sst): old prealloc_size = 147639500, real_size = 32605248 2024/11/23-04:48:43.639816 140024944858688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245355.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928627 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0306 GB avg-val = 116.53 avg-zval = 32.95 2024/11/23-04:48:43.642329 140024944858688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245355.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:48:44.568321 140024944858688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245355.sst): second pass time = 1.33's, 81.153'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 39.512'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3185.388'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.292'MB/sec write SST data time = 0.13's, 276.534'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30389.960'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.153'MB/sec zip pipeline throughput = 81.153'MB/sec entries = 928627 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.53 avg-zval = 32.95 usrkeys = 928627 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.53 avg-zval = 32.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928627 bytes = 3250208 } seq expand size = 6500389 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.4451 value = 3.5371 dict = 1.00 all = 3.6563 } Zip/UnZip{ index = 0.0957 value = 0.2827 dict = 1.00 all = 0.2735 } ---------------------------- total value len = 1.821393 GB avg = 0.121 KB (by entry num) total key len = 0.420123 GB avg = 0.028 KB total ukey len = 0.300088 GB avg = 0.020 KB total ukey num = 0.015004379 Billion total entry num = 0.015004379 Billion write speed all = 0.031226935 MB/sec (with seq num) write speed all = 0.029554706 MB/sec (without seq num) 2024/11/23-04:48:44.602011 140024944858688 [compaction_job.cc:1948] [default] [JOB 19588] Generated table #245355: 928627 keys, 36709104 bytes, temperature: kWarm 2024/11/23-04:48:44.602072 140024944858688 EVENT_LOG_v1 {"time_micros": 1732308524602041, "cf_name": "default", "job": 19588, "event": "table_file_creation", "file_number": 245355, "file_size": 36709104, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3539676816, "largest_seqno": 3793637396, "table_properties": {"data_size": 30594800, "index_size": 1778112, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26001556, "raw_average_key_size": 28, "raw_value_size": 108216210, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928627, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732308521, "oldest_key_time": 0, "file_creation_time": 1732308521, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 245355, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:48:44.636806 140024944858688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245356.sst) /dev/shm/ToplingTemp/Topling-412846-0jcruH 2024/11/23-04:48:44.927935 140024944858688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245356.sst): first pass time = 0.29's, 253.141'MB/sec 2024/11/23-04:48:45.672751 140024944858688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245356.sst): old prealloc_size = 147639500, real_size = 19373680 2024/11/23-04:48:45.673098 140024944858688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245356.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 512299 prefix = 4 raw-key = 0.0082 GB zip-key = 0.0010 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0593 GB zip-val = 0.0183 GB avg-val = 115.84 avg-zval = 35.65 2024/11/23-04:48:45.674591 140024944858688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245356.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.074 GB 2024/11/23-04:48:46.179186 140024944858688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00/245356.sst): second pass time = 0.74's, 79.746'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.51's, 42.850'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3116.564'MB/sec (index lex order gen) rebuild zvType time = 0.43's, 0.296'MB/sec write SST data time = 0.07's, 296.143'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.594'MB, throughput = 17826.285'MB/sec zip my value time = 0.74's, unzip length = 0.059'GB zip my value throughput = 79.746'MB/sec zip pipeline throughput = 79.746'MB/sec entries = 512299 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.84 avg-zval = 35.65 usrkeys = 512299 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.84 avg-zval = 35.65 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 512299 bytes = 1857104 } seq expand size = 3586093 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0102 GB value = 0.0593 GB dict = 0.59 MB all = 0.0737 GB } __ZipSize{ index = 0.0010 GB value = 0.0183 GB dict = 0.59 MB all = 0.0217 GB } UnZip/Zip{ index = 10.4370 value = 3.2492 dict = 1.00 all = 3.3957 } Zip/UnZip{ index = 0.0958 value = 0.3078 dict = 1.00 all = 0.2945 } ---------------------------- total value len = 1.884322 GB avg = 0.121 KB (by entry num) total key len = 0.434467 GB avg = 0.028 KB total ukey len = 0.310334 GB avg = 0.020 KB total ukey num = 0.015516678 Billion total entry num = 0.015516678 Billion write speed all = 0.032302720 MB/sec (with seq num) write speed all = 0.030573435 MB/sec (without seq num) 2024/11/23-04:48:46.202851 140024944858688 [compaction_job.cc:1948] [default] [JOB 19588] Generated table #245356: 512299 keys, 21700055 bytes, temperature: kWarm 2024/11/23-04:48:46.202912 140024944858688 EVENT_LOG_v1 {"time_micros": 1732308526202880, "cf_name": "default", "job": 19588, "event": "table_file_creation", "file_number": 245356, "file_size": 21700055, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3485447251, "largest_seqno": 3793636628, "table_properties": {"data_size": 18263712, "index_size": 981696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 14344372, "raw_average_key_size": 28, "raw_value_size": 59343006, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 512299, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732308524, "oldest_key_time": 0, "file_creation_time": 1732308524, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 245356, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:48:46.221931 140024944858688 [compaction_job.cc:703] job-19588: subcompact[0], size: 0.606601 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19588/att-00 [245340,245341,245342,245343,245344,245345,245346,245347,245348,245349,245350,245351,245352,245353,245354,245355,245356] 2024/11/23-04:48:46.324374 140024944858688 (Original Log Time 2024/11/23-04:47:52.938444) EVENT_LOG_v1 {"time_micros": 1732308472938415, "job": 19588, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [245273], "files_L3": [245323, 245324, 245215, 245216, 245217, 245218, 245219, 245220, 245221, 245222, 245223, 245224, 245225, 245226, 245227, 245242, 245333], "score": 0.999009, "input_data_size": 639182941, "oldest_snapshot_seqno": 3794152805} 2024/11/23-04:48:46.324378 140024944858688 (Original Log Time 2024/11/23-04:48:46.323539) [compaction_job.cc:2022] [default] [JOB 19588] Compacted 1@2 + 17@3 files to L3 => 606601382 bytes 2024/11/23-04:48:46.324379 140024944858688 (Original Log Time 2024/11/23-04:48:46.324305) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 17 0 0 0] max score 0.02, MB/sec: 12.0 rd, 11.4 wr, level 3, files in(1, 17) out(17 +0 blob) MB in(27.1, 582.5 +0.0 blob) out(578.5 +0.0 blob), read-write-amplify(43.8) write-amplify(21.3) OK, records in: 15610119, records dropped: 93441 output_compression: Snappy 2024/11/23-04:48:46.324385 140024944858688 (Original Log Time 2024/11/23-04:48:46.324354) EVENT_LOG_v1 {"time_micros": 1732308526324328, "cf": "default", "job": 19588, "event": "compaction_finished", "compaction_time_micros": 53283494, "compaction_time_cpu_micros": 23956688, "output_level": 3, "num_output_files": 17, "total_output_size": 606601382, "num_input_records": 15610119, "num_output_records": 15516678, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 381762423, "file_range_sync_nanos": 0, "file_fsync_nanos": 15723, "file_prepare_write_nanos": 42157, "lsm_state": [0, 0, 0, 17, 0, 0, 0]} 2024/11/23-04:48:46.456768 140024944858688 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732308472,"executesMs":53511,"compactionJobId":19588,"attempt":0,"compactionInputRawBytes":2234440515,"compactionInputZipBytes":639182941,"compactionOutputRawBytes":0,"compactionOutputZipBytes":606601382,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-69e19221d5ddc4df36596a290c94cb90-7dbd65b0a135bb7a-00"}}, response = 2024/11/23-04:48:46.456837 140024944858688 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-19588/att-00: olev 3, work 53.512 s, result 6.374 ms, install 0.863 ms, input{raw 2.081 GiB zip 609.572 MiB}