2024/11/24-03:32:11.366151 139957618013760 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00: fork to child time = 0.007666 sec 2024/11/24-03:32:11.366917 139957618013760 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00 2024/11/24-03:32:11.374383 139957618013760 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00 2024/11/24-03:32:11.375294 139957618013760 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-03:32:11.375554 139957618013760 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-03:32:11.425343 139957618013760 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-40867/att-00/MANIFEST-206347 2024/11/24-03:32:11.452966 139957618013760 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-40867/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 290471, last_sequence is 4419183209, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 290314 2024/11/24-03:32:11.452991 139957618013760 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-03:32:11.453130 139957618013760 [version_set.cc:5721] Creating manifest 290471 2024/11/24-03:32:11.489052 139957618013760 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-03:32:11.508953 139957618013760 [compaction_job.cc:2437] [default] [JOB 40867] Compacting 1@2 + 20@3 files to L3, score 0.96, subcompactions 1 : 1 2024/11/24-03:32:11.508967 139957618013760 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 30606 Base level 2, inputs: [290414(27MB)], [290450(34MB) 290451(3903KB) 290365(33MB) 290366(4050KB) 290367(33MB) 290368(35MB) 290369(35MB) 290370(34MB) 290371(33MB) 290372(35MB) 290373(35MB) 290374(35MB) 290375(33MB) 290376(34MB) 290377(35MB) 290378(35MB) 290379(33MB) 290380(32MB) 290381(30MB) 290452(34MB)] 2024/11/24-03:32:11.515544 139957618013760 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-03:32:11.515573 139957618013760 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-03:32:11.515593 139957618013760 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-03:32:11.515599 139957618013760 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-03:32:11.515609 139957618013760 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-03:32:12.358329 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290472.sst) /dev/shm/ToplingTemp/Topling-259380-f7yWck 2024/11/24-03:32:12.899349 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290472.sst): first pass time = 0.54's, 248.105'MB/sec 2024/11/24-03:32:14.240198 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290472.sst): old prealloc_size = 147639500, real_size = 31874672 2024/11/24-03:32:14.240733 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290472.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926746 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0299 GB avg-val = 116.83 avg-zval = 32.24 2024/11/24-03:32:14.241572 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290472.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:32:15.065306 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290472.sst): second pass time = 1.34's, 80.840'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.599'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3429.363'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.332'MB/sec write SST data time = 0.13's, 283.489'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30932.688'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.840'MB/sec zip pipeline throughput = 80.840'MB/sec entries = 926746 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.83 avg-zval = 32.24 usrkeys = 926746 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.83 avg-zval = 32.24 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926746 bytes = 3243632 } seq expand size = 6487222 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.5121 value = 3.6235 dict = 1.00 all = 3.7311 } Zip/UnZip{ index = 0.0951 value = 0.2760 dict = 1.00 all = 0.2680 } ---------------------------- total value len = 2251.440329 GB avg = 0.200 KB (by entry num) total key len = 306.908911 GB avg = 0.027 KB total ukey len = 237.373262 GB avg = 0.015 KB total ukey num = 15.739587909 Billion total entry num = 11.280901857 Billion write speed all = 38.171298533 MB/sec (with seq num) write speed all = 36.824784411 MB/sec (without seq num) 2024/11/24-03:32:15.112063 139957618013760 [compaction_job.cc:1948] [default] [JOB 40867] Generated table #290472: 926746 keys, 35972818 bytes, temperature: kWarm 2024/11/24-03:32:15.112177 139957618013760 EVENT_LOG_v1 {"time_micros": 1732390335112112, "cf_name": "default", "job": 40867, "event": "table_file_creation", "file_number": 290472, "file_size": 35972818, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178599, "largest_seqno": 4418795523, "table_properties": {"data_size": 29879584, "index_size": 1763200, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25948888, "raw_average_key_size": 28, "raw_value_size": 108268876, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926746, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390332, "oldest_key_time": 0, "file_creation_time": 1732390332, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290472, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:32:15.149247 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290473.sst) /dev/shm/ToplingTemp/Topling-259380-UGKsvh 2024/11/24-03:32:16.106486 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290473.sst): first pass time = 0.96's, 140.221'MB/sec 2024/11/24-03:32:17.377729 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290473.sst): old prealloc_size = 147639500, real_size = 31307440 2024/11/24-03:32:17.378290 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290473.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921416 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0293 GB avg-val = 117.66 avg-zval = 31.81 2024/11/24-03:32:17.379078 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290473.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:32:18.190472 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290473.sst): second pass time = 1.27's, 85.345'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.545'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3261.484'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.12's, 285.950'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 33970.538'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 85.345'MB/sec zip pipeline throughput = 85.345'MB/sec entries = 921416 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.66 avg-zval = 31.81 usrkeys = 921416 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.66 avg-zval = 31.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921416 bytes = 3224976 } seq expand size = 6449912 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.08 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4448 value = 3.6987 dict = 1.00 all = 3.7925 } Zip/UnZip{ index = 0.0957 value = 0.2704 dict = 1.00 all = 0.2637 } ---------------------------- total value len = 2251.555197 GB avg = 0.200 KB (by entry num) total key len = 306.934711 GB avg = 0.027 KB total ukey len = 237.391691 GB avg = 0.015 KB total ukey num = 15.740509325 Billion total entry num = 11.281823273 Billion write speed all = 38.171617456 MB/sec (with seq num) write speed all = 36.825056140 MB/sec (without seq num) 2024/11/24-03:32:18.234789 139957618013760 [compaction_job.cc:1948] [default] [JOB 40867] Generated table #290473: 921416 keys, 35389894 bytes, temperature: kWarm 2024/11/24-03:32:18.234861 139957618013760 EVENT_LOG_v1 {"time_micros": 1732390338234825, "cf_name": "default", "job": 40867, "event": "table_file_creation", "file_number": 290473, "file_size": 35389894, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4278633053, "largest_seqno": 4418791306, "table_properties": {"data_size": 29312544, "index_size": 1764352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25799648, "raw_average_key_size": 28, "raw_value_size": 108418212, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921416, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390335, "oldest_key_time": 0, "file_creation_time": 1732390335, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290473, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:32:18.270391 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290474.sst) /dev/shm/ToplingTemp/Topling-259380-iTFfBj 2024/11/24-03:32:18.799435 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290474.sst): first pass time = 0.53's, 53.402'MB/sec 2024/11/24-03:32:19.069939 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290474.sst): old prealloc_size = 147639500, real_size = 7375072 2024/11/24-03:32:19.070134 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290474.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 198945 prefix = 4 raw-key = 0.0032 GB zip-key = 0.0004 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0227 GB zip-val = 0.0069 GB avg-val = 114.00 avg-zval = 34.90 2024/11/24-03:32:19.070335 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290474.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0004 GB, waited 0.000 sec, Key+Value = 0.028 GB 2024/11/24-03:32:19.246109 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290474.sst): second pass time = 0.27's, 84.046'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 46.471'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2180.042'MB/sec (index lex order gen) rebuild zvType time = 0.15's, 0.334'MB/sec write SST data time = 0.03's, 302.323'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.237'MB, throughput = 7679.790'MB/sec zip my value time = 0.27's, unzip length = 0.023'GB zip my value throughput = 84.046'MB/sec zip pipeline throughput = 84.046'MB/sec entries = 198945 avg-key = 28.00 avg-zkey = 1.92 avg-val = 114.00 avg-zval = 34.90 usrkeys = 198945 avg-key = 20.00 avg-zkey = 1.92 avg-val = 121.00 avg-zval = 34.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 198945 bytes = 621712 } seq expand size = 1392615 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0040 GB value = 0.0227 GB dict = 0.24 MB all = 0.0283 GB } __ZipSize{ index = 0.0004 GB value = 0.0069 GB dict = 0.24 MB all = 0.0082 GB } UnZip/Zip{ index = 10.4173 value = 3.2665 dict = 1.00 all = 3.4512 } Zip/UnZip{ index = 0.0960 value = 0.3061 dict = 1.00 all = 0.2898 } ---------------------------- total value len = 2251.579270 GB avg = 0.200 KB (by entry num) total key len = 306.940281 GB avg = 0.027 KB total ukey len = 237.395669 GB avg = 0.015 KB total ukey num = 15.740708270 Billion total entry num = 11.282022218 Billion write speed all = 38.171458526 MB/sec (with seq num) write speed all = 36.824894673 MB/sec (without seq num) 2024/11/24-03:32:19.262697 139957618013760 [compaction_job.cc:1948] [default] [JOB 40867] Generated table #290474: 198945 keys, 8185645 bytes, temperature: kWarm 2024/11/24-03:32:19.262750 139957618013760 EVENT_LOG_v1 {"time_micros": 1732390339262722, "cf_name": "default", "job": 40867, "event": "table_file_creation", "file_number": 290474, "file_size": 8185645, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4389782025, "largest_seqno": 4418793509, "table_properties": {"data_size": 6943184, "index_size": 381952, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 5570460, "raw_average_key_size": 28, "raw_value_size": 22679730, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 198945, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390338, "oldest_key_time": 0, "file_creation_time": 1732390338, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290474, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:32:19.294721 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290475.sst) /dev/shm/ToplingTemp/Topling-259380-6zUEwj 2024/11/24-03:32:20.308546 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290475.sst): first pass time = 1.01's, 132.391'MB/sec 2024/11/24-03:32:21.594260 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290475.sst): old prealloc_size = 147639500, real_size = 31398672 2024/11/24-03:32:21.594792 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290475.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.99 2024/11/24-03:32:21.595557 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290475.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:32:22.405339 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290475.sst): second pass time = 1.28's, 84.423'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.740'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3463.128'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.12's, 285.531'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 35030.833'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.423'MB/sec zip pipeline throughput = 84.423'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.99 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4458 value = 3.6886 dict = 1.00 all = 3.7834 } Zip/UnZip{ index = 0.0957 value = 0.2711 dict = 1.00 all = 0.2643 } ---------------------------- total value len = 2251.694182 GB avg = 0.200 KB (by entry num) total key len = 306.966022 GB avg = 0.027 KB total ukey len = 237.414055 GB avg = 0.015 KB total ukey num = 15.741627570 Billion total entry num = 11.282941518 Billion write speed all = 38.171757801 MB/sec (with seq num) write speed all = 36.825147695 MB/sec (without seq num) 2024/11/24-03:32:22.450594 139957618013760 [compaction_job.cc:1948] [default] [JOB 40867] Generated table #290475: 919300 keys, 35475378 bytes, temperature: kWarm 2024/11/24-03:32:22.450668 139957618013760 EVENT_LOG_v1 {"time_micros": 1732390342450629, "cf_name": "default", "job": 40867, "event": "table_file_creation", "file_number": 290475, "file_size": 35475378, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178645, "largest_seqno": 4414357721, "table_properties": {"data_size": 29408512, "index_size": 1760128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390339, "oldest_key_time": 0, "file_creation_time": 1732390339, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290475, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:32:22.504905 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290476.sst) /dev/shm/ToplingTemp/Topling-259380-ceAChk 2024/11/24-03:32:23.043257 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290476.sst): first pass time = 0.54's, 249.330'MB/sec 2024/11/24-03:32:24.347457 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290476.sst): old prealloc_size = 147639500, real_size = 33090288 2024/11/24-03:32:24.348037 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290476.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926740 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 116.83 avg-zval = 33.55 2024/11/24-03:32:24.348794 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290476.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:32:25.205489 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290476.sst): second pass time = 1.30's, 83.063'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.342'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3170.120'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 290.736'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32584.746'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.063'MB/sec zip pipeline throughput = 83.063'MB/sec entries = 926740 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.83 avg-zval = 33.55 usrkeys = 926740 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.83 avg-zval = 33.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926740 bytes = 3243600 } seq expand size = 6487180 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5109 value = 3.4819 dict = 1.00 all = 3.6092 } Zip/UnZip{ index = 0.0951 value = 0.2872 dict = 1.00 all = 0.2771 } ---------------------------- total value len = 2251.808939 GB avg = 0.200 KB (by entry num) total key len = 306.991971 GB avg = 0.027 KB total ukey len = 237.432590 GB avg = 0.015 KB total ukey num = 15.742554310 Billion total entry num = 11.283868258 Billion write speed all = 38.172262304 MB/sec (with seq num) write speed all = 36.825597849 MB/sec (without seq num) 2024/11/24-03:32:25.251883 139957618013760 [compaction_job.cc:1948] [default] [JOB 40867] Generated table #290476: 926740 keys, 37188102 bytes, temperature: kWarm 2024/11/24-03:32:25.251954 139957618013760 EVENT_LOG_v1 {"time_micros": 1732390345251918, "cf_name": "default", "job": 40867, "event": "table_file_creation", "file_number": 290476, "file_size": 37188102, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178655, "largest_seqno": 4418794763, "table_properties": {"data_size": 31095008, "index_size": 1763392, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25948720, "raw_average_key_size": 28, "raw_value_size": 108269096, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926740, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390342, "oldest_key_time": 0, "file_creation_time": 1732390342, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290476, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:32:25.289964 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290477.sst) /dev/shm/ToplingTemp/Topling-259380-e3NrEg 2024/11/24-03:32:26.182429 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290477.sst): first pass time = 0.89's, 150.404'MB/sec 2024/11/24-03:32:27.543337 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290477.sst): old prealloc_size = 147639500, real_size = 33322864 2024/11/24-03:32:27.543848 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290477.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926480 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.87 avg-zval = 33.81 2024/11/24-03:32:27.544635 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290477.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:32:28.393636 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290477.sst): second pass time = 1.36's, 79.609'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.010'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3604.649'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.12's, 301.143'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32783.495'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.609'MB/sec zip pipeline throughput = 79.609'MB/sec entries = 926480 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.87 avg-zval = 33.81 usrkeys = 926480 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.87 avg-zval = 33.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926480 bytes = 3242688 } seq expand size = 6485360 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5137 value = 3.4562 dict = 1.00 all = 3.5868 } Zip/UnZip{ index = 0.0951 value = 0.2893 dict = 1.00 all = 0.2788 } ---------------------------- total value len = 2251.923700 GB avg = 0.200 KB (by entry num) total key len = 307.017912 GB avg = 0.027 KB total ukey len = 237.451120 GB avg = 0.015 KB total ukey num = 15.743480790 Billion total entry num = 11.284794738 Billion write speed all = 38.172545798 MB/sec (with seq num) write speed all = 36.825834824 MB/sec (without seq num) 2024/11/24-03:32:28.440969 139957618013760 [compaction_job.cc:1948] [default] [JOB 40867] Generated table #290477: 926480 keys, 37420336 bytes, temperature: kWarm 2024/11/24-03:32:28.441079 139957618013760 EVENT_LOG_v1 {"time_micros": 1732390348441015, "cf_name": "default", "job": 40867, "event": "table_file_creation", "file_number": 290477, "file_size": 37420336, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178675, "largest_seqno": 4418794780, "table_properties": {"data_size": 31328608, "index_size": 1762432, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25941440, "raw_average_key_size": 28, "raw_value_size": 108276428, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926480, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390345, "oldest_key_time": 0, "file_creation_time": 1732390345, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290477, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:32:28.480960 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290478.sst) /dev/shm/ToplingTemp/Topling-259380-z6Jdfk 2024/11/24-03:32:29.446317 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290478.sst): first pass time = 0.97's, 139.041'MB/sec 2024/11/24-03:32:30.761856 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290478.sst): old prealloc_size = 147639500, real_size = 31583984 2024/11/24-03:32:30.762415 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290478.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926597 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0296 GB avg-val = 116.85 avg-zval = 31.93 2024/11/24-03:32:30.763204 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290478.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:32:31.602863 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290478.sst): second pass time = 1.31's, 82.357'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.429'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3293.505'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.13's, 284.941'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33622.133'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.357'MB/sec zip pipeline throughput = 82.357'MB/sec entries = 926597 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.85 avg-zval = 31.93 usrkeys = 926597 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.85 avg-zval = 31.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926597 bytes = 3243104 } seq expand size = 6486179 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5119 value = 3.6592 dict = 1.00 all = 3.7615 } Zip/UnZip{ index = 0.0951 value = 0.2733 dict = 1.00 all = 0.2658 } ---------------------------- total value len = 2252.038460 GB avg = 0.200 KB (by entry num) total key len = 307.043857 GB avg = 0.027 KB total ukey len = 237.469652 GB avg = 0.015 KB total ukey num = 15.744407387 Billion total entry num = 11.285721335 Billion write speed all = 38.172817278 MB/sec (with seq num) write speed all = 36.826060198 MB/sec (without seq num) 2024/11/24-03:32:31.648021 139957618013760 [compaction_job.cc:1948] [default] [JOB 40867] Generated table #290478: 926597 keys, 35681676 bytes, temperature: kWarm 2024/11/24-03:32:31.648101 139957618013760 EVENT_LOG_v1 {"time_micros": 1732390351648057, "cf_name": "default", "job": 40867, "event": "table_file_creation", "file_number": 290478, "file_size": 35681676, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178686, "largest_seqno": 4418795659, "table_properties": {"data_size": 29589216, "index_size": 1762944, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25944716, "raw_average_key_size": 28, "raw_value_size": 108273150, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926597, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390348, "oldest_key_time": 0, "file_creation_time": 1732390348, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290478, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:32:31.684562 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290479.sst) /dev/shm/ToplingTemp/Topling-259380-MlaPrj 2024/11/24-03:32:32.595933 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290479.sst): first pass time = 0.91's, 147.276'MB/sec 2024/11/24-03:32:33.893750 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290479.sst): old prealloc_size = 147639500, real_size = 31464320 2024/11/24-03:32:33.894326 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290479.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920332 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0295 GB avg-val = 117.84 avg-zval = 32.02 2024/11/24-03:32:33.895082 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290479.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:32:34.710121 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290479.sst): second pass time = 1.30's, 83.608'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.540'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3193.170'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.12's, 287.978'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 34784.009'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.608'MB/sec zip pipeline throughput = 83.608'MB/sec entries = 920332 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.84 avg-zval = 32.02 usrkeys = 920332 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.84 avg-zval = 32.02 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 920332 bytes = 3221184 } seq expand size = 6442324 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4435 value = 3.6798 dict = 1.00 all = 3.7761 } Zip/UnZip{ index = 0.0958 value = 0.2718 dict = 1.00 all = 0.2648 } ---------------------------- total value len = 2252.153350 GB avg = 0.200 KB (by entry num) total key len = 307.069626 GB avg = 0.027 KB total ukey len = 237.488058 GB avg = 0.015 KB total ukey num = 15.745327719 Billion total entry num = 11.286641667 Billion write speed all = 38.173146137 MB/sec (with seq num) write speed all = 36.826341655 MB/sec (without seq num) 2024/11/24-03:32:34.754083 139957618013760 [compaction_job.cc:1948] [default] [JOB 40867] Generated table #290479: 920332 keys, 35543556 bytes, temperature: kWarm 2024/11/24-03:32:34.754151 139957618013760 EVENT_LOG_v1 {"time_micros": 1732390354754117, "cf_name": "default", "job": 40867, "event": "table_file_creation", "file_number": 290479, "file_size": 35543556, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4271870873, "largest_seqno": 4418794816, "table_properties": {"data_size": 29471536, "index_size": 1762496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25769296, "raw_average_key_size": 28, "raw_value_size": 108448444, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920332, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390351, "oldest_key_time": 0, "file_creation_time": 1732390351, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290479, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:32:34.792338 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290480.sst) /dev/shm/ToplingTemp/Topling-259380-fWLUVj 2024/11/24-03:32:35.815419 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290480.sst): first pass time = 1.02's, 131.194'MB/sec 2024/11/24-03:32:37.148464 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290480.sst): old prealloc_size = 147639500, real_size = 33130224 2024/11/24-03:32:37.148973 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290480.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925698 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 116.99 avg-zval = 33.64 2024/11/24-03:32:37.150062 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290480.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:32:37.992142 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290480.sst): second pass time = 1.33's, 81.314'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.123'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3418.081'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.12's, 298.330'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 37552.620'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.314'MB/sec zip pipeline throughput = 81.314'MB/sec entries = 925698 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.99 avg-zval = 33.64 usrkeys = 925698 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.99 avg-zval = 33.64 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925698 bytes = 3239952 } seq expand size = 6479886 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5139 value = 3.4780 dict = 1.00 all = 3.6056 } Zip/UnZip{ index = 0.0951 value = 0.2875 dict = 1.00 all = 0.2773 } ---------------------------- total value len = 2252.268129 GB avg = 0.200 KB (by entry num) total key len = 307.095546 GB avg = 0.027 KB total ukey len = 237.506572 GB avg = 0.015 KB total ukey num = 15.746253417 Billion total entry num = 11.287567365 Billion write speed all = 38.173376013 MB/sec (with seq num) write speed all = 36.826527005 MB/sec (without seq num) 2024/11/24-03:32:38.037946 139957618013760 [compaction_job.cc:1948] [default] [JOB 40867] Generated table #290480: 925698 keys, 37224718 bytes, temperature: kWarm 2024/11/24-03:32:38.038023 139957618013760 EVENT_LOG_v1 {"time_micros": 1732390358037988, "cf_name": "default", "job": 40867, "event": "table_file_creation", "file_number": 290480, "file_size": 37224718, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178700, "largest_seqno": 4418793767, "table_properties": {"data_size": 31137696, "index_size": 1760896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25919544, "raw_average_key_size": 28, "raw_value_size": 108298260, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925698, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390354, "oldest_key_time": 0, "file_creation_time": 1732390354, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290480, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:32:38.073516 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290481.sst) /dev/shm/ToplingTemp/Topling-259380-criF2h 2024/11/24-03:32:39.092310 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290481.sst): first pass time = 1.02's, 131.746'MB/sec 2024/11/24-03:32:40.410606 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290481.sst): old prealloc_size = 147639500, real_size = 33485536 2024/11/24-03:32:40.411181 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290481.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926607 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 116.85 avg-zval = 33.97 2024/11/24-03:32:40.412256 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290481.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:32:41.268573 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290481.sst): second pass time = 1.32's, 82.182'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.806'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3209.613'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.316'MB/sec write SST data time = 0.12's, 301.219'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 30670.603'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.182'MB/sec zip pipeline throughput = 82.182'MB/sec entries = 926607 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.85 avg-zval = 33.97 usrkeys = 926607 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.85 avg-zval = 33.97 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926607 bytes = 3243136 } seq expand size = 6486249 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.4427 value = 3.4395 dict = 1.00 all = 3.5712 } Zip/UnZip{ index = 0.0958 value = 0.2907 dict = 1.00 all = 0.2800 } ---------------------------- total value len = 2252.382888 GB avg = 0.200 KB (by entry num) total key len = 307.121491 GB avg = 0.027 KB total ukey len = 237.525105 GB avg = 0.015 KB total ukey num = 15.747180024 Billion total entry num = 11.288493972 Billion write speed all = 38.173609150 MB/sec (with seq num) write speed all = 36.826715398 MB/sec (without seq num) 2024/11/24-03:32:41.316798 139957618013760 [compaction_job.cc:1948] [default] [JOB 40867] Generated table #290481: 926607 keys, 37583420 bytes, temperature: kWarm 2024/11/24-03:32:41.316869 139957618013760 EVENT_LOG_v1 {"time_micros": 1732390361316833, "cf_name": "default", "job": 40867, "event": "table_file_creation", "file_number": 290481, "file_size": 37583420, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178719, "largest_seqno": 4418795948, "table_properties": {"data_size": 31479056, "index_size": 1774656, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25944996, "raw_average_key_size": 28, "raw_value_size": 108272770, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926607, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390358, "oldest_key_time": 0, "file_creation_time": 1732390358, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290481, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:32:41.357242 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290482.sst) /dev/shm/ToplingTemp/Topling-259380-SuMfsh 2024/11/24-03:32:42.197291 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290482.sst): first pass time = 0.84's, 159.784'MB/sec 2024/11/24-03:32:43.514370 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290482.sst): old prealloc_size = 147639500, real_size = 32022224 2024/11/24-03:32:43.514899 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290482.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926978 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0300 GB avg-val = 116.79 avg-zval = 32.38 2024/11/24-03:32:43.515700 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290482.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:32:44.345538 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290482.sst): second pass time = 1.32's, 82.250'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.459'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3506.338'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.328'MB/sec write SST data time = 0.12's, 292.266'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33201.856'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.250'MB/sec zip pipeline throughput = 82.250'MB/sec entries = 926978 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.79 avg-zval = 32.38 usrkeys = 926978 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.79 avg-zval = 32.38 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926978 bytes = 3244432 } seq expand size = 6488846 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.4438 value = 3.6069 dict = 1.00 all = 3.7158 } Zip/UnZip{ index = 0.0958 value = 0.2772 dict = 1.00 all = 0.2691 } ---------------------------- total value len = 2252.497639 GB avg = 0.200 KB (by entry num) total key len = 307.147446 GB avg = 0.027 KB total ukey len = 237.543644 GB avg = 0.015 KB total ukey num = 15.748107002 Billion total entry num = 11.289420950 Billion write speed all = 38.173955863 MB/sec (with seq num) write speed all = 36.827013322 MB/sec (without seq num) 2024/11/24-03:32:44.389576 139957618013760 [compaction_job.cc:1948] [default] [JOB 40867] Generated table #290482: 926978 keys, 36120914 bytes, temperature: kWarm 2024/11/24-03:32:44.389646 139957618013760 EVENT_LOG_v1 {"time_micros": 1732390364389611, "cf_name": "default", "job": 40867, "event": "table_file_creation", "file_number": 290482, "file_size": 36120914, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178730, "largest_seqno": 4418795264, "table_properties": {"data_size": 30015104, "index_size": 1775168, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25955384, "raw_average_key_size": 28, "raw_value_size": 108262420, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926978, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390361, "oldest_key_time": 0, "file_creation_time": 1732390361, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290482, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:32:44.429099 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290483.sst) /dev/shm/ToplingTemp/Topling-259380-g0hwHh 2024/11/24-03:32:45.419524 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290483.sst): first pass time = 0.99's, 135.520'MB/sec 2024/11/24-03:32:46.737202 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290483.sst): old prealloc_size = 147639500, real_size = 31292768 2024/11/24-03:32:46.737732 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290483.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921534 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0293 GB avg-val = 117.65 avg-zval = 31.79 2024/11/24-03:32:46.739229 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290483.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:32:47.548080 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290483.sst): second pass time = 1.32's, 82.354'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.628'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3493.287'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.12's, 283.132'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 31539.057'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.354'MB/sec zip pipeline throughput = 82.354'MB/sec entries = 921534 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.65 avg-zval = 31.79 usrkeys = 921534 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.65 avg-zval = 31.79 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921534 bytes = 3225376 } seq expand size = 6450738 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.08 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4465 value = 3.7004 dict = 1.00 all = 3.7941 } Zip/UnZip{ index = 0.0957 value = 0.2702 dict = 1.00 all = 0.2636 } ---------------------------- total value len = 2252.612504 GB avg = 0.200 KB (by entry num) total key len = 307.173249 GB avg = 0.027 KB total ukey len = 237.562075 GB avg = 0.015 KB total ukey num = 15.749028536 Billion total entry num = 11.290342484 Billion write speed all = 38.174230489 MB/sec (with seq num) write speed all = 36.827242334 MB/sec (without seq num) 2024/11/24-03:32:47.594036 139957618013760 [compaction_job.cc:1948] [default] [JOB 40867] Generated table #290483: 921534 keys, 35375410 bytes, temperature: kWarm 2024/11/24-03:32:47.594119 139957618013760 EVENT_LOG_v1 {"time_micros": 1732390367594080, "cf_name": "default", "job": 40867, "event": "table_file_creation", "file_number": 290483, "file_size": 35375410, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4279595220, "largest_seqno": 4418794858, "table_properties": {"data_size": 29297936, "index_size": 1764288, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25802952, "raw_average_key_size": 28, "raw_value_size": 108414888, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921534, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390364, "oldest_key_time": 0, "file_creation_time": 1732390364, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290483, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:32:47.648028 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290484.sst) /dev/shm/ToplingTemp/Topling-259380-v6MMTg 2024/11/24-03:32:48.666900 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290484.sst): first pass time = 1.02's, 131.740'MB/sec 2024/11/24-03:32:49.992959 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290484.sst): old prealloc_size = 147639500, real_size = 32835008 2024/11/24-03:32:49.993557 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290484.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924415 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0308 GB avg-val = 117.19 avg-zval = 33.37 2024/11/24-03:32:49.994326 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290484.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:32:50.837908 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290484.sst): second pass time = 1.32's, 81.773'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.704'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3147.434'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 292.830'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 25638.373'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.773'MB/sec zip pipeline throughput = 81.773'MB/sec entries = 924415 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.19 avg-zval = 33.37 usrkeys = 924415 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.19 avg-zval = 33.37 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924415 bytes = 3235472 } seq expand size = 6470905 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5124 value = 3.5122 dict = 1.00 all = 3.6347 } Zip/UnZip{ index = 0.0951 value = 0.2847 dict = 1.00 all = 0.2751 } ---------------------------- total value len = 2252.727310 GB avg = 0.200 KB (by entry num) total key len = 307.199133 GB avg = 0.027 KB total ukey len = 237.580563 GB avg = 0.015 KB total ukey num = 15.749952951 Billion total entry num = 11.291266899 Billion write speed all = 38.174455692 MB/sec (with seq num) write speed all = 36.827423337 MB/sec (without seq num) 2024/11/24-03:32:50.884254 139957618013760 [compaction_job.cc:1948] [default] [JOB 40867] Generated table #290484: 924415 keys, 36926524 bytes, temperature: kWarm 2024/11/24-03:32:50.884325 139957618013760 EVENT_LOG_v1 {"time_micros": 1732390370884290, "cf_name": "default", "job": 40867, "event": "table_file_creation", "file_number": 290484, "file_size": 36926524, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680186, "largest_seqno": 4418791806, "table_properties": {"data_size": 30844976, "index_size": 1758720, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25883620, "raw_average_key_size": 28, "raw_value_size": 108334162, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924415, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390367, "oldest_key_time": 0, "file_creation_time": 1732390367, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290484, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:32:50.925681 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290485.sst) /dev/shm/ToplingTemp/Topling-259380-pzDdTj 2024/11/24-03:32:51.830316 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290485.sst): first pass time = 0.90's, 148.373'MB/sec 2024/11/24-03:32:53.188150 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290485.sst): old prealloc_size = 147639500, real_size = 33418720 2024/11/24-03:32:53.188719 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290485.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926726 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.83 avg-zval = 33.91 2024/11/24-03:32:53.189498 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290485.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:32:54.047998 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290485.sst): second pass time = 1.36's, 79.786'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.634'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3245.941'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.12's, 302.052'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33836.783'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.786'MB/sec zip pipeline throughput = 79.786'MB/sec entries = 926726 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.83 avg-zval = 33.91 usrkeys = 926726 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.83 avg-zval = 33.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926726 bytes = 3243552 } seq expand size = 6487082 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5126 value = 3.4455 dict = 1.00 all = 3.5775 } Zip/UnZip{ index = 0.0951 value = 0.2902 dict = 1.00 all = 0.2795 } ---------------------------- total value len = 2252.842066 GB avg = 0.200 KB (by entry num) total key len = 307.225081 GB avg = 0.027 KB total ukey len = 237.599098 GB avg = 0.015 KB total ukey num = 15.750879677 Billion total entry num = 11.292193625 Billion write speed all = 38.174726514 MB/sec (with seq num) write speed all = 36.827648086 MB/sec (without seq num) 2024/11/24-03:32:54.095317 139957618013760 [compaction_job.cc:1948] [default] [JOB 40867] Generated table #290485: 926726 keys, 37516878 bytes, temperature: kWarm 2024/11/24-03:32:54.095390 139957618013760 EVENT_LOG_v1 {"time_micros": 1732390374095354, "cf_name": "default", "job": 40867, "event": "table_file_creation", "file_number": 290485, "file_size": 37516878, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680199, "largest_seqno": 4418794871, "table_properties": {"data_size": 31423760, "index_size": 1763072, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25948328, "raw_average_key_size": 28, "raw_value_size": 108269404, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926726, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390370, "oldest_key_time": 0, "file_creation_time": 1732390370, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290485, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:32:54.131645 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290486.sst) /dev/shm/ToplingTemp/Topling-259380-LX89bh 2024/11/24-03:32:55.157105 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290486.sst): first pass time = 1.03's, 130.890'MB/sec 2024/11/24-03:32:56.491808 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290486.sst): old prealloc_size = 147639500, real_size = 32936480 2024/11/24-03:32:56.492411 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290486.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926455 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 116.87 avg-zval = 33.40 2024/11/24-03:32:56.494089 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290486.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:32:57.362323 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290486.sst): second pass time = 1.33's, 81.203'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.544'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3039.361'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.312'MB/sec write SST data time = 0.13's, 290.589'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35079.191'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.203'MB/sec zip pipeline throughput = 81.203'MB/sec entries = 926455 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.87 avg-zval = 33.40 usrkeys = 926455 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.87 avg-zval = 33.40 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926455 bytes = 3242608 } seq expand size = 6485185 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5153 value = 3.4993 dict = 1.00 all = 3.6242 } Zip/UnZip{ index = 0.0951 value = 0.2858 dict = 1.00 all = 0.2759 } ---------------------------- total value len = 2252.956828 GB avg = 0.199 KB (by entry num) total key len = 307.251022 GB avg = 0.027 KB total ukey len = 237.617627 GB avg = 0.015 KB total ukey num = 15.751806132 Billion total entry num = 11.293120080 Billion write speed all = 38.174937938 MB/sec (with seq num) write speed all = 36.827815569 MB/sec (without seq num) 2024/11/24-03:32:57.410259 139957618013760 [compaction_job.cc:1948] [default] [JOB 40867] Generated table #290486: 926455 keys, 37033636 bytes, temperature: kWarm 2024/11/24-03:32:57.410335 139957618013760 EVENT_LOG_v1 {"time_micros": 1732390377410295, "cf_name": "default", "job": 40867, "event": "table_file_creation", "file_number": 290486, "file_size": 37033636, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680212, "largest_seqno": 4418795507, "table_properties": {"data_size": 30942544, "index_size": 1762112, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25940740, "raw_average_key_size": 28, "raw_value_size": 108277082, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926455, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390374, "oldest_key_time": 0, "file_creation_time": 1732390374, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290486, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:32:57.452826 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290487.sst) /dev/shm/ToplingTemp/Topling-259380-JnULli 2024/11/24-03:32:58.416564 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290487.sst): first pass time = 0.96's, 139.275'MB/sec 2024/11/24-03:32:59.688507 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290487.sst): old prealloc_size = 147639500, real_size = 30805776 2024/11/24-03:32:59.689149 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290487.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925473 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0288 GB avg-val = 117.03 avg-zval = 31.12 2024/11/24-03:32:59.691361 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290487.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:33:00.509677 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290487.sst): second pass time = 1.27's, 85.204'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.359'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2873.367'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 272.578'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 29894.297'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 85.204'MB/sec zip pipeline throughput = 85.204'MB/sec entries = 925473 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.03 avg-zval = 31.12 usrkeys = 925473 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.03 avg-zval = 31.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925473 bytes = 3123488 } seq expand size = 6478311 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0288 GB dict = 1.08 MB all = 0.0348 GB } UnZip/Zip{ index = 10.4442 value = 3.7603 dict = 1.00 all = 3.8587 } Zip/UnZip{ index = 0.0957 value = 0.2659 dict = 1.00 all = 0.2592 } ---------------------------- total value len = 2253.071611 GB avg = 0.199 KB (by entry num) total key len = 307.276935 GB avg = 0.027 KB total ukey len = 237.636136 GB avg = 0.015 KB total ukey num = 15.752731605 Billion total entry num = 11.294045553 Billion write speed all = 38.175244283 MB/sec (with seq num) write speed all = 36.828074739 MB/sec (without seq num) 2024/11/24-03:33:00.552870 139957618013760 [compaction_job.cc:1948] [default] [JOB 40867] Generated table #290487: 925473 keys, 34782758 bytes, temperature: kWarm 2024/11/24-03:33:00.552944 139957618013760 EVENT_LOG_v1 {"time_micros": 1732390380552907, "cf_name": "default", "job": 40867, "event": "table_file_creation", "file_number": 290487, "file_size": 34782758, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4302851840, "largest_seqno": 4418794900, "table_properties": {"data_size": 28801984, "index_size": 1772224, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25913244, "raw_average_key_size": 28, "raw_value_size": 108304546, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925473, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390377, "oldest_key_time": 0, "file_creation_time": 1732390377, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290487, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:33:00.591036 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290488.sst) /dev/shm/ToplingTemp/Topling-259380-CyS3wh 2024/11/24-03:33:01.546548 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290488.sst): first pass time = 0.96's, 140.475'MB/sec 2024/11/24-03:33:02.881150 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290488.sst): old prealloc_size = 147639500, real_size = 31259744 2024/11/24-03:33:02.881675 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290488.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928973 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0292 GB avg-val = 116.48 avg-zval = 31.48 2024/11/24-03:33:02.882453 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290488.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:33:03.703399 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290488.sst): second pass time = 1.33's, 81.148'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.015'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3540.653'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.333'MB/sec write SST data time = 0.12's, 283.325'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 29749.979'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.148'MB/sec zip pipeline throughput = 81.148'MB/sec entries = 928973 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.48 avg-zval = 31.48 usrkeys = 928973 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.48 avg-zval = 31.48 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928973 bytes = 3251424 } seq expand size = 6502811 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4437 value = 3.6996 dict = 1.00 all = 3.7949 } Zip/UnZip{ index = 0.0958 value = 0.2703 dict = 1.00 all = 0.2635 } ---------------------------- total value len = 2253.186320 GB avg = 0.199 KB (by entry num) total key len = 307.302946 GB avg = 0.027 KB total ukey len = 237.654716 GB avg = 0.015 KB total ukey num = 15.753660578 Billion total entry num = 11.294974526 Billion write speed all = 38.175524577 MB/sec (with seq num) write speed all = 36.828308376 MB/sec (without seq num) 2024/11/24-03:33:03.747872 139957618013760 [compaction_job.cc:1948] [default] [JOB 40867] Generated table #290488: 928973 keys, 35367942 bytes, temperature: kWarm 2024/11/24-03:33:03.747941 139957618013760 EVENT_LOG_v1 {"time_micros": 1732390383747906, "cf_name": "default", "job": 40867, "event": "table_file_creation", "file_number": 290488, "file_size": 35367942, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4418795151, "table_properties": {"data_size": 29248336, "index_size": 1779008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26011244, "raw_average_key_size": 28, "raw_value_size": 108206486, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928973, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390380, "oldest_key_time": 0, "file_creation_time": 1732390380, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290488, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:33:03.784600 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290489.sst) /dev/shm/ToplingTemp/Topling-259380-athfWg 2024/11/24-03:33:04.727105 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290489.sst): first pass time = 0.94's, 142.411'MB/sec 2024/11/24-03:33:05.972657 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290489.sst): old prealloc_size = 147639500, real_size = 27708016 2024/11/24-03:33:05.973253 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290489.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 945196 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1078 GB zip-val = 0.0257 GB avg-val = 114.00 avg-zval = 27.15 2024/11/24-03:33:05.974117 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290489.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:33:06.809031 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290489.sst): second pass time = 1.24's, 86.562'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 37.950'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3167.533'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.334'MB/sec write SST data time = 0.13's, 248.762'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.073'MB, throughput = 35110.195'MB/sec zip my value time = 1.24's, unzip length = 0.108'GB zip my value throughput = 86.562'MB/sec zip pipeline throughput = 86.562'MB/sec entries = 945196 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.00 avg-zval = 27.15 usrkeys = 945196 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.00 avg-zval = 27.15 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 945196 bytes = 3190048 } seq expand size = 6616372 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0257 GB dict = 1.07 MB all = 0.0317 GB } UnZip/Zip{ index = 10.4454 value = 4.1990 dict = 1.00 all = 4.2288 } Zip/UnZip{ index = 0.0957 value = 0.2382 dict = 1.00 all = 0.2365 } ---------------------------- total value len = 2253.300689 GB avg = 0.199 KB (by entry num) total key len = 307.329412 GB avg = 0.027 KB total ukey len = 237.673620 GB avg = 0.015 KB total ukey num = 15.754605774 Billion total entry num = 11.295919722 Billion write speed all = 38.175856671 MB/sec (with seq num) write speed all = 36.828590115 MB/sec (without seq num) 2024/11/24-03:33:06.850793 139957618013760 [compaction_job.cc:1948] [default] [JOB 40867] Generated table #290489: 945196 keys, 31739248 bytes, temperature: kWarm 2024/11/24-03:33:06.850865 139957618013760 EVENT_LOG_v1 {"time_micros": 1732390386850829, "cf_name": "default", "job": 40867, "event": "table_file_creation", "file_number": 290489, "file_size": 31739248, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4297264125, "largest_seqno": 4398742396, "table_properties": {"data_size": 25661728, "index_size": 1809792, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26465488, "raw_average_key_size": 28, "raw_value_size": 107752344, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 945196, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390383, "oldest_key_time": 0, "file_creation_time": 1732390383, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290489, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:33:06.888198 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290490.sst) /dev/shm/ToplingTemp/Topling-259380-hUr0Sj 2024/11/24-03:33:07.896357 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290490.sst): first pass time = 1.01's, 133.137'MB/sec 2024/11/24-03:33:09.238798 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290490.sst): old prealloc_size = 147639500, real_size = 32956464 2024/11/24-03:33:09.239414 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290490.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924382 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 117.20 avg-zval = 33.50 2024/11/24-03:33:09.241475 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290490.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:33:10.087469 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290490.sst): second pass time = 1.34's, 80.784'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.655'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2986.558'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 289.570'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34055.743'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.784'MB/sec zip pipeline throughput = 80.784'MB/sec entries = 924382 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.20 avg-zval = 33.50 usrkeys = 924382 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.20 avg-zval = 33.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924382 bytes = 3235344 } seq expand size = 6470674 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5116 value = 3.4985 dict = 1.00 all = 3.6228 } Zip/UnZip{ index = 0.0951 value = 0.2858 dict = 1.00 all = 0.2760 } ---------------------------- total value len = 2253.415495 GB avg = 0.199 KB (by entry num) total key len = 307.355294 GB avg = 0.027 KB total ukey len = 237.692107 GB avg = 0.015 KB total ukey num = 15.755530156 Billion total entry num = 11.296844104 Billion write speed all = 38.176088207 MB/sec (with seq num) write speed all = 36.828777253 MB/sec (without seq num) 2024/11/24-03:33:10.133690 139957618013760 [compaction_job.cc:1948] [default] [JOB 40867] Generated table #290490: 924382 keys, 37047980 bytes, temperature: kWarm 2024/11/24-03:33:10.133764 139957618013760 EVENT_LOG_v1 {"time_micros": 1732390390133725, "cf_name": "default", "job": 40867, "event": "table_file_creation", "file_number": 290490, "file_size": 37047980, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4418795891, "table_properties": {"data_size": 30966432, "index_size": 1758784, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25882696, "raw_average_key_size": 28, "raw_value_size": 108335052, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924382, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390386, "oldest_key_time": 0, "file_creation_time": 1732390386, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290490, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:33:10.181204 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290491.sst) /dev/shm/ToplingTemp/Topling-259380-EveKFj 2024/11/24-03:33:10.247926 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290491.sst): first pass time = 0.07's, 237.057'MB/sec 2024/11/24-03:33:10.389228 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290491.sst): old prealloc_size = 147639500, real_size = 4801600 2024/11/24-03:33:10.389344 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290491.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 108265 prefix = 4 raw-key = 0.0017 GB zip-key = 0.0002 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0128 GB zip-val = 0.0046 GB avg-val = 118.00 avg-zval = 42.17 2024/11/24-03:33:10.389468 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290491.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0002 GB, waited 0.000 sec, Key+Value = 0.016 GB 2024/11/24-03:33:10.485081 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00/290491.sst): second pass time = 0.14's, 90.873'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.10's, 54.762'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2248.846'MB/sec (index lex order gen) rebuild zvType time = 0.08's, 0.338'MB/sec write SST data time = 0.02's, 334.285'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.133'MB, throughput = 4342.615'MB/sec zip my value time = 0.14's, unzip length = 0.013'GB zip my value throughput = 90.873'MB/sec zip pipeline throughput = 90.873'MB/sec entries = 108265 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 42.17 usrkeys = 108265 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 42.17 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 108265 bytes = 338336 } seq expand size = 757855 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0022 GB value = 0.0128 GB dict = 0.13 MB all = 0.0158 GB } __ZipSize{ index = 0.0002 GB value = 0.0046 GB dict = 0.13 MB all = 0.0052 GB } UnZip/Zip{ index = 10.3813 value = 2.7981 dict = 1.00 all = 3.0122 } Zip/UnZip{ index = 0.0963 value = 0.3574 dict = 1.00 all = 0.3320 } ---------------------------- total value len = 2253.429028 GB avg = 0.199 KB (by entry num) total key len = 307.358326 GB avg = 0.027 KB total ukey len = 237.694272 GB avg = 0.015 KB total ukey num = 15.755638421 Billion total entry num = 11.296952369 Billion write speed all = 38.176108857 MB/sec (with seq num) write speed all = 36.828792977 MB/sec (without seq num) 2024/11/24-03:33:10.498840 139957618013760 [compaction_job.cc:1948] [default] [JOB 40867] Generated table #290491: 108265 keys, 5247611 bytes, temperature: kWarm 2024/11/24-03:33:10.498898 139957618013760 EVENT_LOG_v1 {"time_micros": 1732390390498869, "cf_name": "default", "job": 40867, "event": "table_file_creation", "file_number": 290491, "file_size": 5247611, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4379546534, "largest_seqno": 4396951816, "table_properties": {"data_size": 4565744, "index_size": 208576, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3031420, "raw_average_key_size": 28, "raw_value_size": 12775270, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 108265, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390390, "oldest_key_time": 0, "file_creation_time": 1732390390, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290491, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:33:10.510171 139957618013760 [compaction_job.cc:703] job-40867: subcompact[0], size: 0.662824 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40867/att-00 [290472,290473,290474,290475,290476,290477,290478,290479,290480,290481,290482,290483,290484,290485,290486,290487,290488,290489,290490,290491] 2024/11/24-03:33:10.621187 139957618013760 (Original Log Time 2024/11/24-03:32:11.509036) EVENT_LOG_v1 {"time_micros": 1732390331508995, "job": 40867, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [290414], "files_L3": [290450, 290451, 290365, 290366, 290367, 290368, 290369, 290370, 290371, 290372, 290373, 290374, 290375, 290376, 290377, 290378, 290379, 290380, 290381, 290452], "score": 0.960843, "input_data_size": 685410632, "oldest_snapshot_seqno": 4419183087} 2024/11/24-03:33:10.621191 139957618013760 (Original Log Time 2024/11/24-03:33:10.619528) [compaction_job.cc:2022] [default] [JOB 40867] Compacted 1@2 + 20@3 files to L3 => 662824444 bytes 2024/11/24-03:33:10.621192 139957618013760 (Original Log Time 2024/11/24-03:33:10.621117) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 20 0 0 0] max score 0.02, MB/sec: 11.6 rd, 11.2 wr, level 3, files in(1, 20) out(20 +0 blob) MB in(27.1, 626.6 +0.0 blob) out(632.1 +0.0 blob), read-write-amplify(47.5) write-amplify(23.3) OK, records in: 17079855, records dropped: 102597 output_compression: Snappy 2024/11/24-03:33:10.621196 139957618013760 (Original Log Time 2024/11/24-03:33:10.621162) EVENT_LOG_v1 {"time_micros": 1732390390621141, "cf": "default", "job": 40867, "event": "compaction_finished", "compaction_time_micros": 59001143, "compaction_time_cpu_micros": 25345314, "output_level": 3, "num_output_files": 20, "total_output_size": 662824444, "num_input_records": 17079855, "num_output_records": 16977258, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 460710766, "file_range_sync_nanos": 0, "file_fsync_nanos": 14628, "file_prepare_write_nanos": 55459, "lsm_state": [0, 0, 0, 20, 0, 0, 0]} 2024/11/24-03:33:10.761165 139957618013760 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732390331,"executesMs":59245,"compactionJobId":40867,"attempt":0,"compactionInputRawBytes":2474546018,"compactionInputZipBytes":685410632,"compactionOutputRawBytes":0,"compactionOutputZipBytes":662824444,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-a8e4cf33d5a4d7b0ed1bd8449eaf7d7e-e6fcef4ac8d86f31-00"}}, response = 2024/11/24-03:33:10.761254 139957618013760 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-40867/att-00: olev 3, work 59.245 s, result 7.771 ms, install 1.683 ms, input{raw 2.305 GiB zip 653.659 MiB}