2024/11/23-06:50:03.497822 140025112712768 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00: fork to child time = 0.029750 sec 2024/11/23-06:50:03.498887 140025112712768 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00 2024/11/23-06:50:03.508498 140025112712768 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00 2024/11/23-06:50:03.510010 140025112712768 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-06:50:03.510382 140025112712768 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-06:50:03.559210 140025112712768 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-21349/att-00/MANIFEST-206347 2024/11/23-06:50:03.589273 140025112712768 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-21349/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 249284, last_sequence is 3844949996, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 249124 2024/11/23-06:50:03.589287 140025112712768 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-06:50:03.589432 140025112712768 [version_set.cc:5721] Creating manifest 249284 2024/11/23-06:50:03.622624 140025112712768 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-06:50:03.644643 140025112712768 [compaction_job.cc:2437] [default] [JOB 21349] Compacting 1@2 + 14@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-06:50:03.644670 140025112712768 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 15989 Base level 2, inputs: [249271(27MB)], [249181(36MB) 249182(35MB) 249183(35MB) 249207(36MB) 249208(32MB) 249209(35MB) 249210(35MB) 249211(33MB) 249212(34MB) 249213(34MB) 249214(35MB) 249215(35MB) 249216(35MB) 249281(35MB)] 2024/11/23-06:50:03.652587 140025112712768 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-06:50:03.652630 140025112712768 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-06:50:03.652658 140025112712768 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-06:50:03.652667 140025112712768 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-06:50:03.652682 140025112712768 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-06:50:04.162054 140025112712768 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249285.sst) /dev/shm/ToplingTemp/Topling-439419-nWpW4X 2024/11/23-06:50:04.988738 140025112712768 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249285.sst): first pass time = 0.83's, 162.370'MB/sec 2024/11/23-06:50:06.769834 140025112712768 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249285.sst): old prealloc_size = 147639500, real_size = 34455856 2024/11/23-06:50:06.770399 140025112712768 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249285.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925145 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0325 GB avg-val = 117.08 avg-zval = 35.09 2024/11/23-06:50:06.771222 140025112712768 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249285.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:50:08.488096 140025112712768 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249285.sst): second pass time = 1.78's, 60.863'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.72's, 22.503'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3223.528'MB/sec (index lex order gen) rebuild zvType time = 1.48's, 0.156'MB/sec write SST data time = 0.24's, 162.329'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32338.220'MB/sec zip my value time = 1.78's, unzip length = 0.108'GB zip my value throughput = 60.863'MB/sec zip pipeline throughput = 60.863'MB/sec entries = 925145 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.08 avg-zval = 35.09 usrkeys = 925145 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.08 avg-zval = 35.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925145 bytes = 3353664 } seq expand size = 6476015 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0387 GB } UnZip/Zip{ index = 10.5115 value = 3.3364 dict = 1.00 all = 3.4713 } Zip/UnZip{ index = 0.0951 value = 0.2997 dict = 1.00 all = 0.2881 } ---------------------------- total value len = 0.114790 GB avg = 0.124 KB (by entry num) total key len = 0.025904 GB avg = 0.028 KB total ukey len = 0.018503 GB avg = 0.020 KB total ukey num = 0.000925145 Billion total entry num = 0.000925145 Billion write speed all = 0.001779461 MB/sec (with seq num) write speed all = 0.001685853 MB/sec (without seq num) 2024/11/23-06:50:08.526083 140025112712768 [compaction_job.cc:1948] [default] [JOB 21349] Generated table #249285: 925145 keys, 38664952 bytes, temperature: kWarm 2024/11/23-06:50:08.526200 140025112712768 EVENT_LOG_v1 {"time_micros": 1732315808526126, "cf_name": "default", "job": 21349, "event": "table_file_creation", "file_number": 249285, "file_size": 38664952, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369194706, "largest_seqno": 3844875590, "table_properties": {"data_size": 32464160, "index_size": 1760256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25904060, "raw_average_key_size": 28, "raw_value_size": 108313738, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925145, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732315804, "oldest_key_time": 0, "file_creation_time": 1732315804, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249285, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:50:08.601468 140025112712768 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249286.sst) /dev/shm/ToplingTemp/Topling-439419-9wUukY 2024/11/23-06:50:09.754316 140025112712768 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249286.sst): first pass time = 1.15's, 116.431'MB/sec 2024/11/23-06:50:11.569697 140025112712768 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249286.sst): old prealloc_size = 147639500, real_size = 33438960 2024/11/23-06:50:11.570236 140025112712768 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249286.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928197 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0314 GB avg-val = 116.60 avg-zval = 33.87 2024/11/23-06:50:11.571036 140025112712768 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249286.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:50:12.528822 140025112712768 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249286.sst): second pass time = 1.81's, 59.644'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 39.263'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3404.307'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.280'MB/sec write SST data time = 0.13's, 287.666'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33061.809'MB/sec zip my value time = 1.81's, unzip length = 0.108'GB zip my value throughput = 59.644'MB/sec zip pipeline throughput = 87.240'MB/sec entries = 928197 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.60 avg-zval = 33.87 usrkeys = 928197 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.60 avg-zval = 33.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928197 bytes = 3364736 } seq expand size = 6497379 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5110 value = 3.4423 dict = 1.00 all = 3.5642 } Zip/UnZip{ index = 0.0951 value = 0.2905 dict = 1.00 all = 0.2806 } ---------------------------- total value len = 0.229515 GB avg = 0.124 KB (by entry num) total key len = 0.051894 GB avg = 0.028 KB total ukey len = 0.037067 GB avg = 0.020 KB total ukey num = 0.001853342 Billion total entry num = 0.001853342 Billion write speed all = 0.003559012 MB/sec (with seq num) write speed all = 0.003371497 MB/sec (without seq num) 2024/11/23-06:50:12.563678 140025112712768 [compaction_job.cc:1948] [default] [JOB 21349] Generated table #249286: 928197 keys, 37657516 bytes, temperature: kWarm 2024/11/23-06:50:12.563740 140025112712768 EVENT_LOG_v1 {"time_micros": 1732315812563709, "cf_name": "default", "job": 21349, "event": "table_file_creation", "file_number": 249286, "file_size": 37657516, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 3844875325, "table_properties": {"data_size": 31440608, "index_size": 1766144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25989516, "raw_average_key_size": 28, "raw_value_size": 108228346, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928197, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732315808, "oldest_key_time": 0, "file_creation_time": 1732315808, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249286, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:50:12.601094 140025112712768 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249287.sst) /dev/shm/ToplingTemp/Topling-439419-SWC2r1 2024/11/23-06:50:13.516641 140025112712768 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249287.sst): first pass time = 0.92's, 146.605'MB/sec 2024/11/23-06:50:14.892068 140025112712768 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249287.sst): old prealloc_size = 147639500, real_size = 33307648 2024/11/23-06:50:14.892648 140025112712768 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249287.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928521 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0313 GB avg-val = 116.55 avg-zval = 33.72 2024/11/23-06:50:14.893408 140025112712768 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249287.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:50:15.865638 140025112712768 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249287.sst): second pass time = 1.37's, 78.748'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.97's, 38.428'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3152.992'MB/sec (index lex order gen) rebuild zvType time = 0.84's, 0.275'MB/sec write SST data time = 0.13's, 290.694'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30455.812'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 78.748'MB/sec zip pipeline throughput = 78.748'MB/sec entries = 928521 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.55 avg-zval = 33.72 usrkeys = 928521 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.55 avg-zval = 33.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928521 bytes = 3249840 } seq expand size = 6499647 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5128 value = 3.4565 dict = 1.00 all = 3.5876 } Zip/UnZip{ index = 0.0951 value = 0.2893 dict = 1.00 all = 0.2787 } ---------------------------- total value len = 0.344234 GB avg = 0.124 KB (by entry num) total key len = 0.077892 GB avg = 0.028 KB total ukey len = 0.055637 GB avg = 0.020 KB total ukey num = 0.002781863 Billion total entry num = 0.002781863 Billion write speed all = 0.005338456 MB/sec (with seq num) write speed all = 0.005057008 MB/sec (without seq num) 2024/11/23-06:50:15.901606 140025112712768 [compaction_job.cc:1948] [default] [JOB 21349] Generated table #249287: 928521 keys, 37411176 bytes, temperature: kWarm 2024/11/23-06:50:15.901670 140025112712768 EVENT_LOG_v1 {"time_micros": 1732315815901636, "cf_name": "default", "job": 21349, "event": "table_file_creation", "file_number": 249287, "file_size": 37411176, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3610601794, "largest_seqno": 3844875762, "table_properties": {"data_size": 31308848, "index_size": 1766464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25998588, "raw_average_key_size": 28, "raw_value_size": 108219210, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928521, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732315812, "oldest_key_time": 0, "file_creation_time": 1732315812, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249287, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:50:15.936536 140025112712768 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249288.sst) /dev/shm/ToplingTemp/Topling-439419-fX1LJY 2024/11/23-06:50:16.866420 140025112712768 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249288.sst): first pass time = 0.93's, 144.344'MB/sec 2024/11/23-06:50:18.220947 140025112712768 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249288.sst): old prealloc_size = 147639500, real_size = 33631616 2024/11/23-06:50:18.221654 140025112712768 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249288.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930593 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0316 GB avg-val = 116.23 avg-zval = 33.99 2024/11/23-06:50:18.222451 140025112712768 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249288.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:50:19.145340 140025112712768 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249288.sst): second pass time = 1.35's, 79.893'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.954'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2610.325'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.294'MB/sec write SST data time = 0.13's, 285.496'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31487.927'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 79.893'MB/sec zip pipeline throughput = 79.893'MB/sec entries = 930593 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.23 avg-zval = 33.99 usrkeys = 930593 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.23 avg-zval = 33.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930593 bytes = 3373408 } seq expand size = 6514151 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.5126 value = 3.4198 dict = 1.00 all = 3.5454 } Zip/UnZip{ index = 0.0951 value = 0.2924 dict = 1.00 all = 0.2821 } ---------------------------- total value len = 0.458910 GB avg = 0.124 KB (by entry num) total key len = 0.103949 GB avg = 0.028 KB total ukey len = 0.074249 GB avg = 0.020 KB total ukey num = 0.003712456 Billion total entry num = 0.003712456 Billion write speed all = 0.007117939 MB/sec (with seq num) write speed all = 0.006742356 MB/sec (without seq num) 2024/11/23-06:50:19.180819 140025112712768 [compaction_job.cc:1948] [default] [JOB 21349] Generated table #249288: 930593 keys, 37856622 bytes, temperature: kWarm 2024/11/23-06:50:19.180881 140025112712768 EVENT_LOG_v1 {"time_micros": 1732315819180851, "cf_name": "default", "job": 21349, "event": "table_file_creation", "file_number": 249288, "file_size": 37856622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 3844875379, "table_properties": {"data_size": 31628336, "index_size": 1770432, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26056604, "raw_average_key_size": 28, "raw_value_size": 108161254, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930593, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732315815, "oldest_key_time": 0, "file_creation_time": 1732315815, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249288, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:50:19.215903 140025112712768 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249289.sst) /dev/shm/ToplingTemp/Topling-439419-oSTzOY 2024/11/23-06:50:20.113892 140025112712768 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249289.sst): first pass time = 0.90's, 149.471'MB/sec 2024/11/23-06:50:21.445886 140025112712768 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249289.sst): old prealloc_size = 147639500, real_size = 30776336 2024/11/23-06:50:21.446454 140025112712768 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249289.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926554 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0288 GB avg-val = 116.86 avg-zval = 31.05 2024/11/23-06:50:21.447232 140025112712768 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249289.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:50:22.299827 140025112712768 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249289.sst): second pass time = 1.33's, 81.330'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 40.838'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3243.112'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 273.358'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33661.797'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.330'MB/sec zip pipeline throughput = 81.330'MB/sec entries = 926554 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.86 avg-zval = 31.05 usrkeys = 926554 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.86 avg-zval = 31.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926554 bytes = 3242960 } seq expand size = 6485878 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0288 GB dict = 1.08 MB all = 0.0349 GB } UnZip/Zip{ index = 10.4466 value = 3.7634 dict = 1.00 all = 3.8488 } Zip/UnZip{ index = 0.0957 value = 0.2657 dict = 1.00 all = 0.2598 } ---------------------------- total value len = 0.573670 GB avg = 0.124 KB (by entry num) total key len = 0.129892 GB avg = 0.028 KB total ukey len = 0.092780 GB avg = 0.020 KB total ukey num = 0.004639010 Billion total entry num = 0.004639010 Billion write speed all = 0.008896930 MB/sec (with seq num) write speed all = 0.008427627 MB/sec (without seq num) 2024/11/23-06:50:22.329990 140025112712768 [compaction_job.cc:1948] [default] [JOB 21349] Generated table #249289: 926554 keys, 34872344 bytes, temperature: kWarm 2024/11/23-06:50:22.330054 140025112712768 EVENT_LOG_v1 {"time_micros": 1732315822330021, "cf_name": "default", "job": 21349, "event": "table_file_creation", "file_number": 249289, "file_size": 34872344, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3705219771, "largest_seqno": 3844875390, "table_properties": {"data_size": 28770624, "index_size": 1773888, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25943512, "raw_average_key_size": 28, "raw_value_size": 108274236, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926554, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732315819, "oldest_key_time": 0, "file_creation_time": 1732315819, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249289, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:50:22.364841 140025112712768 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249290.sst) /dev/shm/ToplingTemp/Topling-439419-yHHTu0 2024/11/23-06:50:23.396006 140025112712768 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249290.sst): first pass time = 1.03's, 130.167'MB/sec 2024/11/23-06:50:24.744703 140025112712768 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249290.sst): old prealloc_size = 147639500, real_size = 34312096 2024/11/23-06:50:24.745244 140025112712768 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249290.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923309 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0323 GB avg-val = 117.37 avg-zval = 35.00 2024/11/23-06:50:24.745967 140025112712768 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249290.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:50:25.622085 140025112712768 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249290.sst): second pass time = 1.35's, 80.414'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 43.418'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3385.717'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.309'MB/sec write SST data time = 0.13's, 293.934'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 31619.977'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.414'MB/sec zip pipeline throughput = 80.414'MB/sec entries = 923309 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.37 avg-zval = 35.00 usrkeys = 923309 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.98 avg-zval = 35.00 TagRS{ kinds = RS_Key0_TagN bytes = 144360 } TagArray{ size = 740312 bytes = 2776192 } seq expand size = 5182184 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0323 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.4458 value = 3.3536 dict = 1.00 all = 3.5233 } Zip/UnZip{ index = 0.0957 value = 0.2982 dict = 1.00 all = 0.2838 } ---------------------------- total value len = 0.687217 GB avg = 0.124 KB (by entry num) total key len = 0.155745 GB avg = 0.028 KB total ukey len = 0.111246 GB avg = 0.020 KB total ukey num = 0.005562319 Billion total entry num = 0.005562319 Billion write speed all = 0.010659271 MB/sec (with seq num) write speed all = 0.010096586 MB/sec (without seq num) 2024/11/23-06:50:25.656556 140025112712768 [compaction_job.cc:1948] [default] [JOB 21349] Generated table #249290: 923309 keys, 38093884 bytes, temperature: kWarm 2024/11/23-06:50:25.656619 140025112712768 EVENT_LOG_v1 {"time_micros": 1732315825656589, "cf_name": "default", "job": 21349, "event": "table_file_creation", "file_number": 249290, "file_size": 38093884, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3844875933, "table_properties": {"data_size": 32313296, "index_size": 1767808, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25852652, "raw_average_key_size": 28, "raw_value_size": 108365102, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923309, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732315822, "oldest_key_time": 0, "file_creation_time": 1732315822, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249290, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:50:25.690625 140025112712768 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249291.sst) /dev/shm/ToplingTemp/Topling-439419-ce3eB0 2024/11/23-06:50:26.718385 140025112712768 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249291.sst): first pass time = 1.03's, 130.597'MB/sec 2024/11/23-06:50:28.048925 140025112712768 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249291.sst): old prealloc_size = 147639500, real_size = 31827344 2024/11/23-06:50:28.049468 140025112712768 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249291.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 937848 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0298 GB avg-val = 115.11 avg-zval = 31.78 2024/11/23-06:50:28.050257 140025112712768 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249291.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:50:29.004380 140025112712768 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249291.sst): second pass time = 1.33's, 81.177'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 37.758'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3466.510'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.284'MB/sec write SST data time = 0.13's, 281.146'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 32151.609'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.177'MB/sec zip pipeline throughput = 81.177'MB/sec entries = 937848 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.11 avg-zval = 31.78 usrkeys = 937848 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.11 avg-zval = 31.78 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 937848 bytes = 3399712 } seq expand size = 6564936 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0188 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0298 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.5121 value = 3.6217 dict = 1.00 all = 3.7205 } Zip/UnZip{ index = 0.0951 value = 0.2761 dict = 1.00 all = 0.2688 } ---------------------------- total value len = 0.801740 GB avg = 0.123 KB (by entry num) total key len = 0.182005 GB avg = 0.028 KB total ukey len = 0.130003 GB avg = 0.020 KB total ukey num = 0.006500167 Billion total entry num = 0.006500167 Billion write speed all = 0.012438939 MB/sec (with seq num) write speed all = 0.011781409 MB/sec (without seq num) 2024/11/23-06:50:29.035566 140025112712768 [compaction_job.cc:1948] [default] [JOB 21349] Generated table #249291: 937848 keys, 36075366 bytes, temperature: kWarm 2024/11/23-06:50:29.035629 140025112712768 EVENT_LOG_v1 {"time_micros": 1732315829035597, "cf_name": "default", "job": 21349, "event": "table_file_creation", "file_number": 249291, "file_size": 36075366, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 3844875013, "table_properties": {"data_size": 29808384, "index_size": 1784320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26259744, "raw_average_key_size": 28, "raw_value_size": 107958040, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 937848, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732315825, "oldest_key_time": 0, "file_creation_time": 1732315825, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249291, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:50:29.069894 140025112712768 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249292.sst) /dev/shm/ToplingTemp/Topling-439419-0KvnB2 2024/11/23-06:50:30.047731 140025112712768 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249292.sst): first pass time = 0.98's, 137.266'MB/sec 2024/11/23-06:50:31.420428 140025112712768 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249292.sst): old prealloc_size = 147639500, real_size = 31348176 2024/11/23-06:50:31.420958 140025112712768 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249292.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922979 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0294 GB avg-val = 117.42 avg-zval = 31.80 2024/11/23-06:50:31.421748 140025112712768 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249292.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:50:32.268347 140025112712768 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249292.sst): second pass time = 1.37's, 78.990'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 41.929'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3463.619'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 282.083'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 25575.984'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 78.990'MB/sec zip pipeline throughput = 78.990'MB/sec entries = 922979 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.42 avg-zval = 31.80 usrkeys = 922979 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.42 avg-zval = 31.80 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922979 bytes = 3345808 } seq expand size = 6460853 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.08 MB all = 0.0356 GB } UnZip/Zip{ index = 10.4455 value = 3.6925 dict = 1.00 all = 3.7753 } Zip/UnZip{ index = 0.0957 value = 0.2708 dict = 1.00 all = 0.2649 } ---------------------------- total value len = 0.916575 GB avg = 0.123 KB (by entry num) total key len = 0.207848 GB avg = 0.028 KB total ukey len = 0.148463 GB avg = 0.020 KB total ukey num = 0.007423146 Billion total entry num = 0.007423146 Billion write speed all = 0.014217161 MB/sec (with seq num) write speed all = 0.013466298 MB/sec (without seq num) 2024/11/23-06:50:32.298674 140025112712768 [compaction_job.cc:1948] [default] [JOB 21349] Generated table #249292: 922979 keys, 35551972 bytes, temperature: kWarm 2024/11/23-06:50:32.298735 140025112712768 EVENT_LOG_v1 {"time_micros": 1732315832298705, "cf_name": "default", "job": 21349, "event": "table_file_creation", "file_number": 249292, "file_size": 35551972, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 3844874134, "table_properties": {"data_size": 29350016, "index_size": 1767232, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25843412, "raw_average_key_size": 28, "raw_value_size": 108374446, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922979, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732315829, "oldest_key_time": 0, "file_creation_time": 1732315829, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249292, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:50:32.333708 140025112712768 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249293.sst) /dev/shm/ToplingTemp/Topling-439419-n5z3C0 2024/11/23-06:50:33.302395 140025112712768 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249293.sst): first pass time = 0.97's, 138.562'MB/sec 2024/11/23-06:50:34.643290 140025112712768 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249293.sst): old prealloc_size = 147639500, real_size = 32182112 2024/11/23-06:50:34.643858 140025112712768 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249293.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931980 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0302 GB avg-val = 116.01 avg-zval = 32.37 2024/11/23-06:50:34.644637 140025112712768 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249293.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:50:35.575413 140025112712768 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249293.sst): second pass time = 1.34's, 80.672'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 38.938'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3277.073'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.290'MB/sec write SST data time = 0.13's, 285.370'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33632.891'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.672'MB/sec zip pipeline throughput = 80.672'MB/sec entries = 931980 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.01 avg-zval = 32.37 usrkeys = 931980 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.01 avg-zval = 32.37 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931980 bytes = 3261952 } seq expand size = 6523860 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.4441 value = 3.5845 dict = 1.00 all = 3.6980 } Zip/UnZip{ index = 0.0957 value = 0.2790 dict = 1.00 all = 0.2704 } ---------------------------- total value len = 1.031222 GB avg = 0.123 KB (by entry num) total key len = 0.233944 GB avg = 0.028 KB total ukey len = 0.167103 GB avg = 0.020 KB total ukey num = 0.008355126 Billion total entry num = 0.008355126 Billion write speed all = 0.015996023 MB/sec (with seq num) write speed all = 0.015150924 MB/sec (without seq num) 2024/11/23-06:50:35.607619 140025112712768 [compaction_job.cc:1948] [default] [JOB 21349] Generated table #249293: 931980 keys, 36294678 bytes, temperature: kWarm 2024/11/23-06:50:35.607680 140025112712768 EVENT_LOG_v1 {"time_micros": 1732315835607649, "cf_name": "default", "job": 21349, "event": "table_file_creation", "file_number": 249293, "file_size": 36294678, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3604582272, "largest_seqno": 3844874325, "table_properties": {"data_size": 30164240, "index_size": 1784704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26095440, "raw_average_key_size": 28, "raw_value_size": 108122312, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931980, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732315832, "oldest_key_time": 0, "file_creation_time": 1732315832, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249293, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:50:35.642191 140025112712768 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249294.sst) /dev/shm/ToplingTemp/Topling-439419-9Demf2 2024/11/23-06:50:36.574830 140025112712768 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249294.sst): first pass time = 0.93's, 143.918'MB/sec 2024/11/23-06:50:37.966124 140025112712768 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249294.sst): old prealloc_size = 147639500, real_size = 31352944 2024/11/23-06:50:37.966711 140025112712768 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249294.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931093 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0293 GB avg-val = 116.15 avg-zval = 31.52 2024/11/23-06:50:37.967547 140025112712768 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249294.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:50:38.882187 140025112712768 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249294.sst): second pass time = 1.39's, 77.800'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 38.840'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3142.819'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.295'MB/sec write SST data time = 0.13's, 279.260'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 26366.824'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 77.800'MB/sec zip pipeline throughput = 77.800'MB/sec entries = 931093 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.15 avg-zval = 31.52 usrkeys = 931093 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.15 avg-zval = 31.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931093 bytes = 3375232 } seq expand size = 6517651 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.08 MB all = 0.0356 GB } UnZip/Zip{ index = 10.5122 value = 3.6849 dict = 1.00 all = 3.7723 } Zip/UnZip{ index = 0.0951 value = 0.2714 dict = 1.00 all = 0.2651 } ---------------------------- total value len = 1.145886 GB avg = 0.123 KB (by entry num) total key len = 0.260014 GB avg = 0.028 KB total ukey len = 0.185724 GB avg = 0.020 KB total ukey num = 0.009286219 Billion total entry num = 0.009286219 Billion write speed all = 0.017774658 MB/sec (with seq num) write speed all = 0.016835420 MB/sec (without seq num) 2024/11/23-06:50:38.913412 140025112712768 [compaction_job.cc:1948] [default] [JOB 21349] Generated table #249294: 931093 keys, 35579588 bytes, temperature: kWarm 2024/11/23-06:50:38.913476 140025112712768 EVENT_LOG_v1 {"time_micros": 1732315838913444, "cf_name": "default", "job": 21349, "event": "table_file_creation", "file_number": 249294, "file_size": 35579588, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3485447251, "largest_seqno": 3844875134, "table_properties": {"data_size": 29348512, "index_size": 1771456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26070604, "raw_average_key_size": 28, "raw_value_size": 108147162, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931093, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732315835, "oldest_key_time": 0, "file_creation_time": 1732315835, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249294, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:50:38.953468 140025112712768 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249295.sst) /dev/shm/ToplingTemp/Topling-439419-2NGInZ 2024/11/23-06:50:39.855504 140025112712768 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249295.sst): first pass time = 0.90's, 148.801'MB/sec 2024/11/23-06:50:41.208046 140025112712768 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249295.sst): old prealloc_size = 147639500, real_size = 32983872 2024/11/23-06:50:41.208628 140025112712768 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249295.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928994 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.48 avg-zval = 33.34 2024/11/23-06:50:41.209448 140025112712768 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249295.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:50:42.136868 140025112712768 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249295.sst): second pass time = 1.35's, 80.043'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.056'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3153.939'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.290'MB/sec write SST data time = 0.13's, 289.209'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33616.705'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.043'MB/sec zip pipeline throughput = 80.043'MB/sec entries = 928994 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.48 avg-zval = 33.34 usrkeys = 928994 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.48 avg-zval = 33.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928994 bytes = 3367616 } seq expand size = 6502958 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.4428 value = 3.4936 dict = 1.00 all = 3.6076 } Zip/UnZip{ index = 0.0958 value = 0.2862 dict = 1.00 all = 0.2772 } ---------------------------- total value len = 1.260595 GB avg = 0.123 KB (by entry num) total key len = 0.286026 GB avg = 0.028 KB total ukey len = 0.204304 GB avg = 0.020 KB total ukey num = 0.010215213 Billion total entry num = 0.010215213 Billion write speed all = 0.019552971 MB/sec (with seq num) write speed all = 0.018519814 MB/sec (without seq num) 2024/11/23-06:50:42.172574 140025112712768 [compaction_job.cc:1948] [default] [JOB 21349] Generated table #249295: 928994 keys, 37204602 bytes, temperature: kWarm 2024/11/23-06:50:42.172637 140025112712768 EVENT_LOG_v1 {"time_micros": 1732315842172605, "cf_name": "default", "job": 21349, "event": "table_file_creation", "file_number": 249295, "file_size": 37204602, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 3844875201, "table_properties": {"data_size": 30972208, "index_size": 1779200, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26011832, "raw_average_key_size": 28, "raw_value_size": 108205908, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928994, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732315838, "oldest_key_time": 0, "file_creation_time": 1732315838, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249295, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:50:42.227632 140025112712768 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249296.sst) /dev/shm/ToplingTemp/Topling-439419-ThBc8X 2024/11/23-06:50:43.127637 140025112712768 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249296.sst): first pass time = 0.90's, 149.136'MB/sec 2024/11/23-06:50:44.504542 140025112712768 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249296.sst): old prealloc_size = 147639500, real_size = 32286032 2024/11/23-06:50:44.505125 140025112712768 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249296.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928654 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0303 GB avg-val = 116.53 avg-zval = 32.61 2024/11/23-06:50:44.505909 140025112712768 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249296.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:50:45.409344 140025112712768 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249296.sst): second pass time = 1.38's, 78.633'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 40.219'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3143.517'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.299'MB/sec write SST data time = 0.13's, 282.488'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31322.970'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.633'MB/sec zip pipeline throughput = 78.633'MB/sec entries = 928654 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.53 avg-zval = 32.61 usrkeys = 928654 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.53 avg-zval = 32.61 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928654 bytes = 3250304 } seq expand size = 6500578 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5108 value = 3.5730 dict = 1.00 all = 3.6883 } Zip/UnZip{ index = 0.0951 value = 0.2799 dict = 1.00 all = 0.2711 } ---------------------------- total value len = 1.375311 GB avg = 0.123 KB (by entry num) total key len = 0.312028 GB avg = 0.028 KB total ukey len = 0.222877 GB avg = 0.020 KB total ukey num = 0.011143867 Billion total entry num = 0.011143867 Billion write speed all = 0.021331103 MB/sec (with seq num) write speed all = 0.020204069 MB/sec (without seq num) 2024/11/23-06:50:45.443300 140025112712768 [compaction_job.cc:1948] [default] [JOB 21349] Generated table #249296: 928654 keys, 36390048 bytes, temperature: kWarm 2024/11/23-06:50:45.443362 140025112712768 EVENT_LOG_v1 {"time_micros": 1732315845443330, "cf_name": "default", "job": 21349, "event": "table_file_creation", "file_number": 249296, "file_size": 36390048, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3596991077, "largest_seqno": 3844872738, "table_properties": {"data_size": 30286656, "index_size": 1767040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26002312, "raw_average_key_size": 28, "raw_value_size": 108215436, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928654, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732315842, "oldest_key_time": 0, "file_creation_time": 1732315842, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249296, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:50:45.478801 140025112712768 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249297.sst) /dev/shm/ToplingTemp/Topling-439419-WCU1T1 2024/11/23-06:50:46.381316 140025112712768 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249297.sst): first pass time = 0.90's, 148.722'MB/sec 2024/11/23-06:50:47.738229 140025112712768 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249297.sst): old prealloc_size = 147639500, real_size = 32200768 2024/11/23-06:50:47.738890 140025112712768 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249297.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928832 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0302 GB avg-val = 116.50 avg-zval = 32.52 2024/11/23-06:50:47.739726 140025112712768 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249297.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:50:48.687532 140025112712768 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249297.sst): second pass time = 1.36's, 79.789'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 38.366'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2753.918'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.290'MB/sec write SST data time = 0.15's, 244.780'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 24670.475'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.789'MB/sec zip pipeline throughput = 79.789'MB/sec entries = 928832 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.50 avg-zval = 32.52 usrkeys = 928832 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.50 avg-zval = 32.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928832 bytes = 3367024 } seq expand size = 6501824 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5110 value = 3.5830 dict = 1.00 all = 3.6852 } Zip/UnZip{ index = 0.0951 value = 0.2791 dict = 1.00 all = 0.2714 } ---------------------------- total value len = 1.490024 GB avg = 0.123 KB (by entry num) total key len = 0.338036 GB avg = 0.028 KB total ukey len = 0.241454 GB avg = 0.020 KB total ukey num = 0.012072699 Billion total entry num = 0.012072699 Billion write speed all = 0.023109103 MB/sec (with seq num) write speed all = 0.021888183 MB/sec (without seq num) 2024/11/23-06:50:48.725135 140025112712768 [compaction_job.cc:1948] [default] [JOB 21349] Generated table #249297: 928832 keys, 36421120 bytes, temperature: kWarm 2024/11/23-06:50:48.725201 140025112712768 EVENT_LOG_v1 {"time_micros": 1732315848725169, "cf_name": "default", "job": 21349, "event": "table_file_creation", "file_number": 249297, "file_size": 36421120, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 3844874560, "table_properties": {"data_size": 30201008, "index_size": 1767360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26007296, "raw_average_key_size": 28, "raw_value_size": 108210560, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928832, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732315845, "oldest_key_time": 0, "file_creation_time": 1732315845, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249297, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:50:48.768228 140025112712768 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249298.sst) /dev/shm/ToplingTemp/Topling-439419-upCbC1 2024/11/23-06:50:49.619653 140025112712768 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249298.sst): first pass time = 0.85's, 157.645'MB/sec 2024/11/23-06:50:50.962906 140025112712768 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249298.sst): old prealloc_size = 147639500, real_size = 32860944 2024/11/23-06:50:50.963493 140025112712768 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249298.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928432 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.56 avg-zval = 33.23 2024/11/23-06:50:50.964306 140025112712768 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249298.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:50:51.888422 140025112712768 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249298.sst): second pass time = 1.34's, 80.645'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.064'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3141.815'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.292'MB/sec write SST data time = 0.13's, 284.527'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 23652.142'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.645'MB/sec zip pipeline throughput = 80.645'MB/sec entries = 928432 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.56 avg-zval = 33.23 usrkeys = 928432 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.56 avg-zval = 33.23 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928432 bytes = 3365584 } seq expand size = 6499024 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.4455 value = 3.5079 dict = 1.00 all = 3.6197 } Zip/UnZip{ index = 0.0957 value = 0.2851 dict = 1.00 all = 0.2763 } ---------------------------- total value len = 1.604744 GB avg = 0.123 KB (by entry num) total key len = 0.364032 GB avg = 0.028 KB total ukey len = 0.260023 GB avg = 0.020 KB total ukey num = 0.013001131 Billion total entry num = 0.013001131 Billion write speed all = 0.024886943 MB/sec (with seq num) write speed all = 0.023572184 MB/sec (without seq num) 2024/11/23-06:50:51.919808 140025112712768 [compaction_job.cc:1948] [default] [JOB 21349] Generated table #249298: 928432 keys, 37079810 bytes, temperature: kWarm 2024/11/23-06:50:51.919872 140025112712768 EVENT_LOG_v1 {"time_micros": 1732315851919840, "cf_name": "default", "job": 21349, "event": "table_file_creation", "file_number": 249298, "file_size": 37079810, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 3844874692, "table_properties": {"data_size": 30851008, "index_size": 1777664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25996096, "raw_average_key_size": 28, "raw_value_size": 108221688, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732315848, "oldest_key_time": 0, "file_creation_time": 1732315848, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249298, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:50:51.964029 140025112712768 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249299.sst) /dev/shm/ToplingTemp/Topling-439419-SVASvZ 2024/11/23-06:50:52.053101 140025112712768 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249299.sst): first pass time = 0.09's, 239.768'MB/sec 2024/11/23-06:50:52.256132 140025112712768 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249299.sst): old prealloc_size = 147639500, real_size = 6274368 2024/11/23-06:50:52.256268 140025112712768 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249299.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 146215 prefix = 4 raw-key = 0.0023 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0173 GB zip-val = 0.0060 GB avg-val = 118.00 avg-zval = 40.74 2024/11/23-06:50:52.256417 140025112712768 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249299.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.021 GB 2024/11/23-06:50:52.394333 140025112712768 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00/249299.sst): second pass time = 0.20's, 85.215'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.14's, 50.034'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2512.987'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.311'MB/sec write SST data time = 0.02's, 336.886'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.182'MB, throughput = 5957.447'MB/sec zip my value time = 0.20's, unzip length = 0.017'GB zip my value throughput = 85.215'MB/sec zip pipeline throughput = 85.215'MB/sec entries = 146215 avg-key = 28.00 avg-zkey = 1.92 avg-val = 118.00 avg-zval = 40.74 usrkeys = 146215 avg-key = 20.00 avg-zkey = 1.92 avg-val = 125.00 avg-zval = 40.74 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 146215 bytes = 493488 } seq expand size = 1023505 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0029 GB value = 0.0173 GB dict = 0.18 MB all = 0.0213 GB } __ZipSize{ index = 0.0003 GB value = 0.0060 GB dict = 0.18 MB all = 0.0069 GB } UnZip/Zip{ index = 10.3940 value = 2.8967 dict = 1.00 all = 3.0873 } Zip/UnZip{ index = 0.0962 value = 0.3452 dict = 1.00 all = 0.3239 } ---------------------------- total value len = 1.623021 GB avg = 0.123 KB (by entry num) total key len = 0.368126 GB avg = 0.028 KB total ukey len = 0.262947 GB avg = 0.020 KB total ukey num = 0.013147346 Billion total entry num = 0.013147346 Billion write speed all = 0.025169569 MB/sec (with seq num) write speed all = 0.023840032 MB/sec (without seq num) 2024/11/23-06:50:52.405714 140025112712768 [compaction_job.cc:1948] [default] [JOB 21349] Generated table #249299: 146215 keys, 6914615 bytes, temperature: kWarm 2024/11/23-06:50:52.405769 140025112712768 EVENT_LOG_v1 {"time_micros": 1732315852405740, "cf_name": "default", "job": 21349, "event": "table_file_creation", "file_number": 249299, "file_size": 6914615, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3668815374, "largest_seqno": 3794185728, "table_properties": {"data_size": 5956272, "index_size": 281344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4094020, "raw_average_key_size": 28, "raw_value_size": 17253370, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 146215, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732315851, "oldest_key_time": 0, "file_creation_time": 1732315851, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249299, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:50:52.416711 140025112712768 [compaction_job.cc:703] job-21349: subcompact[0], size: 0.522068 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21349/att-00 [249285,249286,249287,249288,249289,249290,249291,249292,249293,249294,249295,249296,249297,249298,249299] 2024/11/23-06:50:52.513269 140025112712768 (Original Log Time 2024/11/23-06:50:03.644765) EVENT_LOG_v1 {"time_micros": 1732315803644693, "job": 21349, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [249271], "files_L3": [249181, 249182, 249183, 249207, 249208, 249209, 249210, 249211, 249212, 249213, 249214, 249215, 249216, 249281], "score": 0.995149, "input_data_size": 545148510, "oldest_snapshot_seqno": 3844949932} 2024/11/23-06:50:52.513273 140025112712768 (Original Log Time 2024/11/23-06:50:52.512477) [compaction_job.cc:2022] [default] [JOB 21349] Compacted 1@2 + 14@3 files to L3 => 522068293 bytes 2024/11/23-06:50:52.513274 140025112712768 (Original Log Time 2024/11/23-06:50:52.513204) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 11.2 rd, 10.7 wr, level 3, files in(1, 14) out(15 +0 blob) MB in(27.1, 492.8 +0.0 blob) out(497.9 +0.0 blob), read-write-amplify(37.6) write-amplify(18.4) OK, records in: 13233010, records dropped: 85664 output_compression: Snappy 2024/11/23-06:50:52.513281 140025112712768 (Original Log Time 2024/11/23-06:50:52.513250) EVENT_LOG_v1 {"time_micros": 1732315852513228, "cf": "default", "job": 21349, "event": "compaction_finished", "compaction_time_micros": 48771951, "compaction_time_cpu_micros": 22427902, "output_level": 3, "num_output_files": 15, "total_output_size": 522068293, "num_input_records": 13233010, "num_output_records": 13147346, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 337040857, "file_range_sync_nanos": 0, "file_fsync_nanos": 13356, "file_prepare_write_nanos": 33956, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/23-06:50:52.655753 140025112712768 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732315803,"executesMs":49008,"compactionJobId":21349,"attempt":0,"compactionInputRawBytes":1912560784,"compactionInputZipBytes":545148510,"compactionOutputRawBytes":0,"compactionOutputZipBytes":522068293,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-c1cd3050b175382ded7e123740f5b68b-bec79f6ded663e87-00"}}, response = 2024/11/23-06:50:52.655834 140025112712768 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-21349/att-00: olev 3, work 49.008 s, result 6.262 ms, install 0.828 ms, input{raw 1.781 GiB zip 519.894 MiB}