2024/11/23-14:51:11.069138 139957685155392 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00: fork to child time = 0.032988 sec 2024/11/23-14:51:11.070165 139957685155392 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00 2024/11/23-14:51:11.080232 139957685155392 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00 2024/11/23-14:51:11.081465 139957685155392 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-14:51:11.081838 139957685155392 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-14:51:11.130782 139957685155392 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28440/att-00/MANIFEST-206347 2024/11/23-14:51:11.149366 139957685155392 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28440/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 264599, last_sequence is 4041182108, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 264444 2024/11/23-14:51:11.149377 139957685155392 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-14:51:11.149564 139957685155392 [version_set.cc:5721] Creating manifest 264599 2024/11/23-14:51:11.184031 139957685155392 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-14:51:11.209266 139957685155392 [compaction_job.cc:2437] [default] [JOB 28440] Compacting 1@2 + 20@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-14:51:11.209305 139957685155392 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 21242 Base level 2, inputs: [264579(31MB)], [264497(40MB) 264498(40MB) 264499(40MB) 264500(40MB) 264501(40MB) 264502(40MB) 264503(40MB) 264504(40MB) 264505(40MB) 264506(40MB) 264507(40MB) 264508(40MB) 264509(39MB) 264510(40MB) 264511(40MB) 264512(40MB) 264513(40MB) 264514(40MB) 264515(39MB) 264596(40MB)] 2024/11/23-14:51:11.218321 139957685155392 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:51:11.218374 139957685155392 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:51:11.218403 139957685155392 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:51:11.218412 139957685155392 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-14:51:11.218432 139957685155392 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-14:51:11.926869 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264600.sst) /dev/shm/ToplingTemp/Topling-81986-l8RO6A 2024/11/23-14:51:12.335300 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264600.sst): first pass time = 0.41's, 328.673'MB/sec 2024/11/23-14:51:14.237890 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264600.sst): old prealloc_size = 147639500, real_size = 40682224 2024/11/23-14:51:14.238081 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264600.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.81 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1016.99 avg-zval = 312.26 2024/11/23-14:51:14.238301 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264600.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:51:14.433870 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264600.sst): second pass time = 1.90's, 69.223'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.20's, 216.924'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1431.475'MB/sec (index lex order gen) rebuild zvType time = 0.15's, 0.221'MB/sec write SST data time = 0.05's, 859.719'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 40026.639'MB/sec zip my value time = 1.90's, unzip length = 0.132'GB zip my value throughput = 69.223'MB/sec zip pipeline throughput = 106.252'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.81 avg-val = 1016.99 avg-zval = 312.26 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.81 avg-val = 1023.99 avg-zval = 312.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129431 bytes = 485376 } seq expand size = 906017 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0425 GB } UnZip/Zip{ index = 6.6470 value = 3.2569 dict = 1.00 all = 3.1570 } Zip/UnZip{ index = 0.1504 value = 0.3070 dict = 1.00 all = 0.3168 } ---------------------------- total value len = 8.981155 GB avg = 0.171 KB (by entry num) total key len = 1.443410 GB avg = 0.028 KB total ukey len = 0.999431 GB avg = 0.015 KB total ukey num = 0.065679866 Billion total entry num = 0.052487595 Billion write speed all = 0.487990731 MB/sec (with seq num) write speed all = 0.468334497 MB/sec (without seq num) 2024/11/23-14:51:14.487162 139957685155392 [compaction_job.cc:1948] [default] [JOB 28440] Generated table #264600: 129431 keys, 42514860 bytes, temperature: kWarm 2024/11/23-14:51:14.487254 139957685155392 EVENT_LOG_v1 {"time_micros": 1732344674487197, "cf_name": "default", "job": 28440, "event": "table_file_creation", "file_number": 264600, "file_size": 42514860, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237877277, "largest_seqno": 4041081841, "table_properties": {"data_size": 40416032, "index_size": 233664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629454, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344671, "oldest_key_time": 0, "file_creation_time": 1732344671, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264600, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:51:14.527470 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264601.sst) /dev/shm/ToplingTemp/Topling-81986-A9qEaB 2024/11/23-14:51:15.183135 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264601.sst): first pass time = 0.66's, 204.721'MB/sec 2024/11/23-14:51:16.249196 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264601.sst): old prealloc_size = 147639500, real_size = 40825248 2024/11/23-14:51:16.249310 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264601.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.76 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1016.99 avg-zval = 313.41 2024/11/23-14:51:16.249464 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264601.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:51:16.415382 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264601.sst): second pass time = 1.07's, 123.557'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 256.724'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2425.979'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.272'MB/sec write SST data time = 0.05's, 909.207'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 38289.556'MB/sec zip my value time = 1.07's, unzip length = 0.132'GB zip my value throughput = 123.557'MB/sec zip pipeline throughput = 123.557'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.76 avg-val = 1016.99 avg-zval = 313.41 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.76 avg-val = 1023.99 avg-zval = 313.41 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129431 bytes = 485376 } seq expand size = 906017 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.8055 value = 3.2449 dict = 1.00 all = 3.1464 } Zip/UnZip{ index = 0.1469 value = 0.3082 dict = 1.00 all = 0.3178 } ---------------------------- total value len = 9.113691 GB avg = 0.173 KB (by entry num) total key len = 1.445998 GB avg = 0.027 KB total ukey len = 1.000984 GB avg = 0.015 KB total ukey num = 0.065809297 Billion total entry num = 0.052617026 Billion write speed all = 0.494270264 MB/sec (with seq num) write speed all = 0.474567387 MB/sec (without seq num) 2024/11/23-14:51:16.467524 139957685155392 [compaction_job.cc:1948] [default] [JOB 28440] Generated table #264601: 129431 keys, 42658155 bytes, temperature: kWarm 2024/11/23-14:51:16.467591 139957685155392 EVENT_LOG_v1 {"time_micros": 1732344676467557, "cf_name": "default", "job": 28440, "event": "table_file_creation", "file_number": 264601, "file_size": 42658155, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868195, "largest_seqno": 4041075357, "table_properties": {"data_size": 40564496, "index_size": 228224, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629526, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344674, "oldest_key_time": 0, "file_creation_time": 1732344674, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264601, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:51:16.506065 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264602.sst) /dev/shm/ToplingTemp/Topling-81986-vxKayB 2024/11/23-14:51:17.460941 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264602.sst): first pass time = 0.95's, 140.567'MB/sec 2024/11/23-14:51:18.948828 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264602.sst): old prealloc_size = 147639500, real_size = 40700352 2024/11/23-14:51:18.949011 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264602.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129433 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1016.97 avg-zval = 312.40 2024/11/23-14:51:18.949212 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264602.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:51:19.274058 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264602.sst): second pass time = 1.49's, 88.515'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.33's, 130.786'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1375.175'MB/sec (index lex order gen) rebuild zvType time = 0.28's, 0.117'MB/sec write SST data time = 0.05's, 871.064'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44200.519'MB/sec zip my value time = 1.49's, unzip length = 0.132'GB zip my value throughput = 88.515'MB/sec zip pipeline throughput = 117.548'MB/sec entries = 129433 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1016.97 avg-zval = 312.40 usrkeys = 129433 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1023.97 avg-zval = 312.40 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129433 bytes = 485392 } seq expand size = 906031 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0425 GB } UnZip/Zip{ index = 6.6636 value = 3.2554 dict = 1.00 all = 3.1556 } Zip/UnZip{ index = 0.1501 value = 0.3072 dict = 1.00 all = 0.3169 } ---------------------------- total value len = 9.246227 GB avg = 0.175 KB (by entry num) total key len = 1.448587 GB avg = 0.027 KB total ukey len = 1.002538 GB avg = 0.015 KB total ukey num = 0.065938730 Billion total entry num = 0.052746459 Billion write speed all = 0.500528101 MB/sec (with seq num) write speed all = 0.480779400 MB/sec (without seq num) 2024/11/23-14:51:19.326139 139957685155392 [compaction_job.cc:1948] [default] [JOB 28440] Generated table #264602: 129433 keys, 42533484 bytes, temperature: kWarm 2024/11/23-14:51:19.326203 139957685155392 EVENT_LOG_v1 {"time_micros": 1732344679326170, "cf_name": "default", "job": 28440, "event": "table_file_creation", "file_number": 264602, "file_size": 42533484, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237870807, "largest_seqno": 4041081842, "table_properties": {"data_size": 40434736, "index_size": 233088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588660, "raw_average_key_size": 20, "raw_value_size": 131629854, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129433, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344676, "oldest_key_time": 0, "file_creation_time": 1732344676, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264602, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:51:19.365598 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264603.sst) /dev/shm/ToplingTemp/Topling-81986-O5XdWA 2024/11/23-14:51:20.175405 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264603.sst): first pass time = 0.81's, 165.750'MB/sec 2024/11/23-14:51:21.073243 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264603.sst): old prealloc_size = 147639500, real_size = 40800880 2024/11/23-14:51:21.073394 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264603.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129435 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.77 raw-val = 0.1316 GB zip-val = 0.0405 GB avg-val =1016.95 avg-zval = 313.20 2024/11/23-14:51:21.073579 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264603.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:51:21.267590 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264603.sst): second pass time = 0.90's, 146.738'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.19's, 219.416'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1863.647'MB/sec (index lex order gen) rebuild zvType time = 0.13's, 0.256'MB/sec write SST data time = 0.07's, 627.127'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 48535.214'MB/sec zip my value time = 0.90's, unzip length = 0.132'GB zip my value throughput = 146.738'MB/sec zip pipeline throughput = 146.738'MB/sec entries = 129435 avg-key = 20.00 avg-zkey = 1.77 avg-val = 1016.95 avg-zval = 313.20 usrkeys = 129435 avg-key = 12.00 avg-zkey = 1.77 avg-val = 1023.95 avg-zval = 313.20 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129435 bytes = 485392 } seq expand size = 906045 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0405 GB dict = 1.38 MB all = 0.0426 GB } UnZip/Zip{ index = 6.7885 value = 3.2469 dict = 1.00 all = 3.1482 } Zip/UnZip{ index = 0.1473 value = 0.3080 dict = 1.00 all = 0.3176 } ---------------------------- total value len = 9.378762 GB avg = 0.177 KB (by entry num) total key len = 1.451176 GB avg = 0.027 KB total ukey len = 1.004091 GB avg = 0.015 KB total ukey num = 0.066068165 Billion total entry num = 0.052875894 Billion write speed all = 0.506804771 MB/sec (with seq num) write speed all = 0.487009455 MB/sec (without seq num) 2024/11/23-14:51:21.319647 139957685155392 [compaction_job.cc:1948] [default] [JOB 28440] Generated table #264603: 129435 keys, 42633982 bytes, temperature: kWarm 2024/11/23-14:51:21.319715 139957685155392 EVENT_LOG_v1 {"time_micros": 1732344681319681, "cf_name": "default", "job": 28440, "event": "table_file_creation", "file_number": 264603, "file_size": 42633982, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868596, "largest_seqno": 4041079292, "table_properties": {"data_size": 40539552, "index_size": 228800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588700, "raw_average_key_size": 20, "raw_value_size": 131629545, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129435, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344679, "oldest_key_time": 0, "file_creation_time": 1732344679, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264603, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:51:21.365829 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264604.sst) /dev/shm/ToplingTemp/Topling-81986-g0InhD 2024/11/23-14:51:22.131761 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264604.sst): first pass time = 0.77's, 175.246'MB/sec 2024/11/23-14:51:23.665674 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264604.sst): old prealloc_size = 147639500, real_size = 40597488 2024/11/23-14:51:23.665808 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264604.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.82 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.00 avg-zval = 311.59 2024/11/23-14:51:23.665989 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264604.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:51:23.853598 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264604.sst): second pass time = 1.53's, 85.919'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.19's, 225.816'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2217.283'MB/sec (index lex order gen) rebuild zvType time = 0.13's, 0.246'MB/sec write SST data time = 0.06's, 751.465'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 36598.438'MB/sec zip my value time = 1.53's, unzip length = 0.132'GB zip my value throughput = 85.919'MB/sec zip pipeline throughput = 126.928'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.82 avg-val = 1017.00 avg-zval = 311.59 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.82 avg-val = 1024.00 avg-zval = 311.59 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.5803 value = 3.2639 dict = 1.00 all = 3.1633 } Zip/UnZip{ index = 0.1520 value = 0.3064 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 9.511298 GB avg = 0.179 KB (by entry num) total key len = 1.453764 GB avg = 0.027 KB total ukey len = 1.005644 GB avg = 0.015 KB total ukey num = 0.066197595 Billion total entry num = 0.053005324 Billion write speed all = 0.513066067 MB/sec (with seq num) write speed all = 0.493224697 MB/sec (without seq num) 2024/11/23-14:51:23.905059 139957685155392 [compaction_job.cc:1948] [default] [JOB 28440] Generated table #264604: 129430 keys, 42430328 bytes, temperature: kWarm 2024/11/23-14:51:23.905120 139957685155392 EVENT_LOG_v1 {"time_micros": 1732344683905088, "cf_name": "default", "job": 28440, "event": "table_file_creation", "file_number": 264604, "file_size": 42430328, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237872175, "largest_seqno": 4041081859, "table_properties": {"data_size": 40328928, "index_size": 236032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131630157, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344681, "oldest_key_time": 0, "file_creation_time": 1732344681, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264604, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:51:23.952167 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264605.sst) /dev/shm/ToplingTemp/Topling-81986-DZWjGC 2024/11/23-14:51:24.589957 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264605.sst): first pass time = 0.64's, 210.455'MB/sec 2024/11/23-14:51:25.454251 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264605.sst): old prealloc_size = 147639500, real_size = 40929632 2024/11/23-14:51:25.454377 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264605.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.76 raw-val = 0.1316 GB zip-val = 0.0407 GB avg-val =1017.01 avg-zval = 314.22 2024/11/23-14:51:25.454530 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264605.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:51:25.618008 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264605.sst): second pass time = 0.86's, 152.427'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 261.190'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2252.262'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.05's, 931.968'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 52572.191'MB/sec zip my value time = 0.86's, unzip length = 0.132'GB zip my value throughput = 152.427'MB/sec zip pipeline throughput = 152.427'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.76 avg-val = 1017.01 avg-zval = 314.22 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.76 avg-val = 1024.01 avg-zval = 314.22 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129429 bytes = 485376 } seq expand size = 906003 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0407 GB dict = 1.38 MB all = 0.0428 GB } UnZip/Zip{ index = 6.8054 value = 3.2366 dict = 1.00 all = 3.1387 } Zip/UnZip{ index = 0.1469 value = 0.3090 dict = 1.00 all = 0.3186 } ---------------------------- total value len = 9.643834 GB avg = 0.181 KB (by entry num) total key len = 1.456353 GB avg = 0.027 KB total ukey len = 1.007197 GB avg = 0.015 KB total ukey num = 0.066327024 Billion total entry num = 0.053134753 Billion write speed all = 0.519345805 MB/sec (with seq num) write speed all = 0.499457628 MB/sec (without seq num) 2024/11/23-14:51:25.669171 139957685155392 [compaction_job.cc:1948] [default] [JOB 28440] Generated table #264605: 129429 keys, 42762563 bytes, temperature: kWarm 2024/11/23-14:51:25.669239 139957685155392 EVENT_LOG_v1 {"time_micros": 1732344685669205, "cf_name": "default", "job": 28440, "event": "table_file_creation", "file_number": 264605, "file_size": 42762563, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237885133, "largest_seqno": 4041069897, "table_properties": {"data_size": 40668880, "index_size": 228224, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131630047, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344683, "oldest_key_time": 0, "file_creation_time": 1732344683, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264605, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:51:25.710647 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264606.sst) /dev/shm/ToplingTemp/Topling-81986-ZN1QoA 2024/11/23-14:51:26.362780 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264606.sst): first pass time = 0.65's, 205.828'MB/sec 2024/11/23-14:51:27.282003 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264606.sst): old prealloc_size = 147639500, real_size = 41001568 2024/11/23-14:51:27.282119 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264606.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.73 raw-val = 0.1316 GB zip-val = 0.0407 GB avg-val =1017.02 avg-zval = 314.81 2024/11/23-14:51:27.282273 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264606.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:51:27.443553 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264606.sst): second pass time = 0.92's, 143.420'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 265.143'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2428.877'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.04's, 979.588'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 51348.575'MB/sec zip my value time = 0.92's, unzip length = 0.132'GB zip my value throughput = 143.420'MB/sec zip pipeline throughput = 143.420'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.73 avg-val = 1017.02 avg-zval = 314.81 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.73 avg-val = 1024.02 avg-zval = 314.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129427 bytes = 485360 } seq expand size = 905989 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0407 GB dict = 1.38 MB all = 0.0428 GB } UnZip/Zip{ index = 6.9376 value = 3.2306 dict = 1.00 all = 3.1334 } Zip/UnZip{ index = 0.1441 value = 0.3095 dict = 1.00 all = 0.3191 } ---------------------------- total value len = 9.776371 GB avg = 0.184 KB (by entry num) total key len = 1.458941 GB avg = 0.027 KB total ukey len = 1.008750 GB avg = 0.015 KB total ukey num = 0.066456451 Billion total entry num = 0.053264180 Billion write speed all = 0.525623005 MB/sec (with seq num) write speed all = 0.505688087 MB/sec (without seq num) 2024/11/23-14:51:27.496881 139957685155392 [compaction_job.cc:1948] [default] [JOB 28440] Generated table #264606: 129427 keys, 42834194 bytes, temperature: kWarm 2024/11/23-14:51:27.496960 139957685155392 EVENT_LOG_v1 {"time_micros": 1732344687496917, "cf_name": "default", "job": 28440, "event": "table_file_creation", "file_number": 264606, "file_size": 42834194, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868290, "largest_seqno": 4041074407, "table_properties": {"data_size": 40745168, "index_size": 223872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131630167, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344685, "oldest_key_time": 0, "file_creation_time": 1732344685, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264606, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:51:27.540164 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264607.sst) /dev/shm/ToplingTemp/Topling-81986-JsQ7Ry 2024/11/23-14:51:28.206367 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264607.sst): first pass time = 0.67's, 201.478'MB/sec 2024/11/23-14:51:29.029589 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264607.sst): old prealloc_size = 147639500, real_size = 40922544 2024/11/23-14:51:29.029717 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264607.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129434 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0407 GB avg-val =1016.96 avg-zval = 314.19 2024/11/23-14:51:29.029872 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264607.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:51:29.190732 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264607.sst): second pass time = 0.82's, 160.212'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 265.366'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2370.169'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.04's, 990.017'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 46111.260'MB/sec zip my value time = 0.82's, unzip length = 0.132'GB zip my value throughput = 160.212'MB/sec zip pipeline throughput = 160.212'MB/sec entries = 129434 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1016.96 avg-zval = 314.19 usrkeys = 129434 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1023.96 avg-zval = 314.19 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129434 bytes = 485392 } seq expand size = 906038 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0407 GB dict = 1.38 MB all = 0.0428 GB } UnZip/Zip{ index = 6.9698 value = 3.2367 dict = 1.00 all = 3.1392 } Zip/UnZip{ index = 0.1435 value = 0.3090 dict = 1.00 all = 0.3186 } ---------------------------- total value len = 9.908906 GB avg = 0.186 KB (by entry num) total key len = 1.461530 GB avg = 0.027 KB total ukey len = 1.010303 GB avg = 0.015 KB total ukey num = 0.066585885 Billion total entry num = 0.053393614 Billion write speed all = 0.531901048 MB/sec (with seq num) write speed all = 0.511919321 MB/sec (without seq num) 2024/11/23-14:51:29.242015 139957685155392 [compaction_job.cc:1948] [default] [JOB 28440] Generated table #264607: 129434 keys, 42755793 bytes, temperature: kWarm 2024/11/23-14:51:29.242086 139957685155392 EVENT_LOG_v1 {"time_micros": 1732344689242049, "cf_name": "default", "job": 28440, "event": "table_file_creation", "file_number": 264607, "file_size": 42755793, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237874784, "largest_seqno": 4041082189, "table_properties": {"data_size": 40667168, "index_size": 222848, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588680, "raw_average_key_size": 20, "raw_value_size": 131629218, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129434, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344687, "oldest_key_time": 0, "file_creation_time": 1732344687, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264607, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:51:29.278716 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264608.sst) /dev/shm/ToplingTemp/Topling-81986-qkUggD 2024/11/23-14:51:29.938762 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264608.sst): first pass time = 0.66's, 203.359'MB/sec 2024/11/23-14:51:30.844960 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264608.sst): old prealloc_size = 147639500, real_size = 40591984 2024/11/23-14:51:30.845093 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264608.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129432 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.82 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.98 avg-zval = 311.54 2024/11/23-14:51:30.845251 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264608.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:51:31.005397 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264608.sst): second pass time = 0.91's, 145.372'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 264.472'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2339.497'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.277'MB/sec write SST data time = 0.04's, 972.994'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 52885.008'MB/sec zip my value time = 0.91's, unzip length = 0.132'GB zip my value throughput = 145.372'MB/sec zip pipeline throughput = 158.520'MB/sec entries = 129432 avg-key = 20.00 avg-zkey = 1.82 avg-val = 1016.98 avg-zval = 311.54 usrkeys = 129432 avg-key = 12.00 avg-zkey = 1.82 avg-val = 1023.98 avg-zval = 311.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129432 bytes = 485392 } seq expand size = 906024 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.5858 value = 3.2643 dict = 1.00 all = 3.1637 } Zip/UnZip{ index = 0.1518 value = 0.3063 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 10.041441 GB avg = 0.188 KB (by entry num) total key len = 1.464119 GB avg = 0.027 KB total ukey len = 1.011857 GB avg = 0.015 KB total ukey num = 0.066715317 Billion total entry num = 0.053523046 Billion write speed all = 0.538176376 MB/sec (with seq num) write speed all = 0.518147911 MB/sec (without seq num) 2024/11/23-14:51:31.056439 139957685155392 [compaction_job.cc:1948] [default] [JOB 28440] Generated table #264608: 129432 keys, 42424649 bytes, temperature: kWarm 2024/11/23-14:51:31.056509 139957685155392 EVENT_LOG_v1 {"time_micros": 1732344691056473, "cf_name": "default", "job": 28440, "event": "table_file_creation", "file_number": 264608, "file_size": 42424649, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237873877, "largest_seqno": 4041079373, "table_properties": {"data_size": 40323616, "index_size": 235840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588640, "raw_average_key_size": 20, "raw_value_size": 131629467, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344689, "oldest_key_time": 0, "file_creation_time": 1732344689, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264608, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:51:31.127688 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264609.sst) /dev/shm/ToplingTemp/Topling-81986-tSPcrA 2024/11/23-14:51:31.801394 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264609.sst): first pass time = 0.67's, 199.237'MB/sec 2024/11/23-14:51:32.595659 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264609.sst): old prealloc_size = 147639500, real_size = 40593968 2024/11/23-14:51:32.595777 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264609.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.82 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.99 avg-zval = 311.56 2024/11/23-14:51:32.595935 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264609.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:51:32.756458 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264609.sst): second pass time = 0.79's, 165.896'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 263.891'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2428.488'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.274'MB/sec write SST data time = 0.04's, 1000.450'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 29522.517'MB/sec zip my value time = 0.79's, unzip length = 0.132'GB zip my value throughput = 165.896'MB/sec zip pipeline throughput = 165.896'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.82 avg-val = 1016.99 avg-zval = 311.56 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.82 avg-val = 1023.99 avg-zval = 311.56 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.5803 value = 3.2642 dict = 1.00 all = 3.1635 } Zip/UnZip{ index = 0.1520 value = 0.3064 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 10.173977 GB avg = 0.190 KB (by entry num) total key len = 1.466707 GB avg = 0.027 KB total ukey len = 1.013410 GB avg = 0.015 KB total ukey num = 0.066844747 Billion total entry num = 0.053652476 Billion write speed all = 0.544452241 MB/sec (with seq num) write speed all = 0.524376987 MB/sec (without seq num) 2024/11/23-14:51:32.808691 139957685155392 [compaction_job.cc:1948] [default] [JOB 28440] Generated table #264609: 129430 keys, 42426760 bytes, temperature: kWarm 2024/11/23-14:51:32.808760 139957685155392 EVENT_LOG_v1 {"time_micros": 1732344692808724, "cf_name": "default", "job": 28440, "event": "table_file_creation", "file_number": 264609, "file_size": 42426760, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237872082, "largest_seqno": 4041075358, "table_properties": {"data_size": 40325408, "index_size": 236032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629175, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344691, "oldest_key_time": 0, "file_creation_time": 1732344691, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264609, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:51:32.844014 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264610.sst) /dev/shm/ToplingTemp/Topling-81986-k7KDaz 2024/11/23-14:51:33.448961 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264610.sst): first pass time = 0.60's, 221.883'MB/sec 2024/11/23-14:51:34.460900 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264610.sst): old prealloc_size = 147639500, real_size = 41050432 2024/11/23-14:51:34.461039 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264610.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.73 raw-val = 0.1316 GB zip-val = 0.0408 GB avg-val =1016.99 avg-zval = 315.18 2024/11/23-14:51:34.461206 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264610.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:51:34.620104 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264610.sst): second pass time = 1.01's, 130.174'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 269.394'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1925.987'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.274'MB/sec write SST data time = 0.04's, 1045.769'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 47165.845'MB/sec zip my value time = 1.01's, unzip length = 0.132'GB zip my value throughput = 130.174'MB/sec zip pipeline throughput = 157.370'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.73 avg-val = 1016.99 avg-zval = 315.18 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.73 avg-val = 1023.99 avg-zval = 315.18 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129431 bytes = 485376 } seq expand size = 906017 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0408 GB dict = 1.38 MB all = 0.0429 GB } UnZip/Zip{ index = 6.9338 value = 3.2267 dict = 1.00 all = 3.1298 } Zip/UnZip{ index = 0.1442 value = 0.3099 dict = 1.00 all = 0.3195 } ---------------------------- total value len = 10.306512 GB avg = 0.192 KB (by entry num) total key len = 1.469296 GB avg = 0.027 KB total ukey len = 1.014963 GB avg = 0.015 KB total ukey num = 0.066974178 Billion total entry num = 0.053781907 Billion write speed all = 0.550724198 MB/sec (with seq num) write speed all = 0.530602268 MB/sec (without seq num) 2024/11/23-14:51:34.671390 139957685155392 [compaction_job.cc:1948] [default] [JOB 28440] Generated table #264610: 129431 keys, 42883401 bytes, temperature: kWarm 2024/11/23-14:51:34.671453 139957685155392 EVENT_LOG_v1 {"time_micros": 1732344694671421, "cf_name": "default", "job": 28440, "event": "table_file_creation", "file_number": 264610, "file_size": 42883401, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237888398, "largest_seqno": 4041078585, "table_properties": {"data_size": 40793904, "index_size": 224000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629543, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344692, "oldest_key_time": 0, "file_creation_time": 1732344692, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264610, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:51:34.712101 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264611.sst) /dev/shm/ToplingTemp/Topling-81986-0MMoEA 2024/11/23-14:51:35.219418 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264611.sst): first pass time = 0.51's, 264.584'MB/sec 2024/11/23-14:51:36.000848 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264611.sst): old prealloc_size = 147639500, real_size = 40652992 2024/11/23-14:51:36.000968 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264611.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129425 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.82 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1017.03 avg-zval = 312.04 2024/11/23-14:51:36.001138 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264611.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:51:36.168076 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264611.sst): second pass time = 0.78's, 168.704'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 254.099'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2134.205'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.267'MB/sec write SST data time = 0.05's, 924.601'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 47572.030'MB/sec zip my value time = 0.78's, unzip length = 0.132'GB zip my value throughput = 168.704'MB/sec zip pipeline throughput = 168.704'MB/sec entries = 129425 avg-key = 20.00 avg-zkey = 1.82 avg-val = 1017.03 avg-zval = 312.04 usrkeys = 129425 avg-key = 12.00 avg-zkey = 1.82 avg-val = 1024.03 avg-zval = 312.04 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129425 bytes = 485360 } seq expand size = 905975 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0425 GB } UnZip/Zip{ index = 6.6105 value = 3.2593 dict = 1.00 all = 3.1591 } Zip/UnZip{ index = 0.1513 value = 0.3068 dict = 1.00 all = 0.3165 } ---------------------------- total value len = 10.439048 GB avg = 0.194 KB (by entry num) total key len = 1.471884 GB avg = 0.027 KB total ukey len = 1.016516 GB avg = 0.015 KB total ukey num = 0.067103603 Billion total entry num = 0.053911332 Billion write speed all = 0.557003282 MB/sec (with seq num) write speed all = 0.536834390 MB/sec (without seq num) 2024/11/23-14:51:36.220932 139957685155392 [compaction_job.cc:1948] [default] [JOB 28440] Generated table #264611: 129425 keys, 42485730 bytes, temperature: kWarm 2024/11/23-14:51:36.221011 139957685155392 EVENT_LOG_v1 {"time_micros": 1732344696220966, "cf_name": "default", "job": 28440, "event": "table_file_creation", "file_number": 264611, "file_size": 42485730, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237871365, "largest_seqno": 4041056803, "table_properties": {"data_size": 40385520, "index_size": 234944, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588500, "raw_average_key_size": 20, "raw_value_size": 131629677, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129425, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344694, "oldest_key_time": 0, "file_creation_time": 1732344694, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264611, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:51:36.271203 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264612.sst) /dev/shm/ToplingTemp/Topling-81986-5eIJwC 2024/11/23-14:51:36.980656 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264612.sst): first pass time = 0.71's, 189.196'MB/sec 2024/11/23-14:51:37.861526 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264612.sst): old prealloc_size = 147639500, real_size = 40453360 2024/11/23-14:51:37.861797 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264612.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.71 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1017.00 avg-zval = 310.59 2024/11/23-14:51:37.861914 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264612.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:51:38.054593 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264612.sst): second pass time = 0.88's, 149.581'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.19's, 217.144'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1753.640'MB/sec (index lex order gen) rebuild zvType time = 0.15's, 0.218'MB/sec write SST data time = 0.04's, 935.761'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45035.815'MB/sec zip my value time = 0.88's, unzip length = 0.132'GB zip my value throughput = 149.581'MB/sec zip pipeline throughput = 177.265'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.71 avg-val = 1017.00 avg-zval = 310.59 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.71 avg-val = 1018.42 avg-zval = 310.59 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 26249 bytes = 98448 } seq expand size = 183743 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0419 GB } UnZip/Zip{ index = 7.0078 value = 3.2744 dict = 1.00 all = 3.2018 } Zip/UnZip{ index = 0.1427 value = 0.3054 dict = 1.00 all = 0.3123 } ---------------------------- total value len = 10.570861 GB avg = 0.196 KB (by entry num) total key len = 1.474473 GB avg = 0.027 KB total ukey len = 1.018069 GB avg = 0.015 KB total ukey num = 0.067233032 Billion total entry num = 0.054040761 Billion write speed all = 0.563238755 MB/sec (with seq num) write speed all = 0.543023225 MB/sec (without seq num) 2024/11/23-14:51:38.105161 139957685155392 [compaction_job.cc:1948] [default] [JOB 28440] Generated table #264612: 129429 keys, 41919689 bytes, temperature: kWarm 2024/11/23-14:51:38.105231 139957685155392 EVENT_LOG_v1 {"time_micros": 1732344698105195, "cf_name": "default", "job": 28440, "event": "table_file_creation", "file_number": 264612, "file_size": 41919689, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4041071449, "table_properties": {"data_size": 40199200, "index_size": 221632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629294, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344696, "oldest_key_time": 0, "file_creation_time": 1732344696, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264612, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:51:38.143153 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264613.sst) /dev/shm/ToplingTemp/Topling-81986-FhYeqA 2024/11/23-14:51:38.855791 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264613.sst): first pass time = 0.71's, 188.348'MB/sec 2024/11/23-14:51:39.693295 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264613.sst): old prealloc_size = 147639500, real_size = 41070864 2024/11/23-14:51:39.693422 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264613.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.71 raw-val = 0.1316 GB zip-val = 0.0408 GB avg-val =1016.99 avg-zval = 315.36 2024/11/23-14:51:39.693578 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264613.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:51:39.855941 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264613.sst): second pass time = 0.84's, 157.297'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 263.682'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2232.956'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.270'MB/sec write SST data time = 0.04's, 1003.042'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 49078.986'MB/sec zip my value time = 0.84's, unzip length = 0.132'GB zip my value throughput = 157.297'MB/sec zip pipeline throughput = 157.297'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.71 avg-val = 1016.99 avg-zval = 315.36 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.71 avg-val = 1023.37 avg-zval = 315.36 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 117849 bytes = 441952 } seq expand size = 824943 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0408 GB dict = 1.38 MB all = 0.0429 GB } UnZip/Zip{ index = 7.0139 value = 3.2249 dict = 1.00 all = 3.1300 } Zip/UnZip{ index = 0.1426 value = 0.3101 dict = 1.00 all = 0.3195 } ---------------------------- total value len = 10.703315 GB avg = 0.198 KB (by entry num) total key len = 1.477062 GB avg = 0.027 KB total ukey len = 1.019622 GB avg = 0.015 KB total ukey num = 0.067362462 Billion total entry num = 0.054170191 Billion write speed all = 0.569505383 MB/sec (with seq num) write speed all = 0.549243143 MB/sec (without seq num) 2024/11/23-14:51:39.908457 139957685155392 [compaction_job.cc:1948] [default] [JOB 28440] Generated table #264613: 129430 keys, 42880714 bytes, temperature: kWarm 2024/11/23-14:51:39.908524 139957685155392 EVENT_LOG_v1 {"time_micros": 1732344699908490, "cf_name": "default", "job": 28440, "event": "table_file_creation", "file_number": 264613, "file_size": 42880714, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4041081229, "table_properties": {"data_size": 40816896, "index_size": 221440, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629446, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344698, "oldest_key_time": 0, "file_creation_time": 1732344698, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264613, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:51:39.946716 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264614.sst) /dev/shm/ToplingTemp/Topling-81986-62tpzB 2024/11/23-14:51:40.554990 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264614.sst): first pass time = 0.61's, 220.667'MB/sec 2024/11/23-14:51:41.369386 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264614.sst): old prealloc_size = 147639500, real_size = 40791696 2024/11/23-14:51:41.369533 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264614.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.74 raw-val = 0.1316 GB zip-val = 0.0405 GB avg-val =1017.02 avg-zval = 313.18 2024/11/23-14:51:41.369703 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264614.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:51:41.595774 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264614.sst): second pass time = 0.81's, 162.005'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.23's, 187.720'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1789.104'MB/sec (index lex order gen) rebuild zvType time = 0.18's, 0.179'MB/sec write SST data time = 0.05's, 930.033'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 35516.838'MB/sec zip my value time = 0.81's, unzip length = 0.132'GB zip my value throughput = 162.005'MB/sec zip pipeline throughput = 162.005'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.74 avg-val = 1017.02 avg-zval = 313.18 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.74 avg-val = 1021.84 avg-zval = 313.18 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 89105 bytes = 334160 } seq expand size = 623735 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0405 GB dict = 1.38 MB all = 0.0425 GB } UnZip/Zip{ index = 6.8962 value = 3.2474 dict = 1.00 all = 3.1585 } Zip/UnZip{ index = 0.1450 value = 0.3079 dict = 1.00 all = 0.3166 } ---------------------------- total value len = 10.835569 GB avg = 0.200 KB (by entry num) total key len = 1.479650 GB avg = 0.027 KB total ukey len = 1.021175 GB avg = 0.015 KB total ukey num = 0.067491889 Billion total entry num = 0.054299618 Billion write speed all = 0.575763209 MB/sec (with seq num) write speed all = 0.555454209 MB/sec (without seq num) 2024/11/23-14:51:41.647992 139957685155392 [compaction_job.cc:1948] [default] [JOB 28440] Generated table #264614: 129427 keys, 42493963 bytes, temperature: kWarm 2024/11/23-14:51:41.648061 139957685155392 EVENT_LOG_v1 {"time_micros": 1732344701648027, "cf_name": "default", "job": 28440, "event": "table_file_creation", "file_number": 264614, "file_size": 42493963, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4041067652, "table_properties": {"data_size": 40533952, "index_size": 225216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629595, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344699, "oldest_key_time": 0, "file_creation_time": 1732344699, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264614, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:51:41.686090 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264615.sst) /dev/shm/ToplingTemp/Topling-81986-cQvaNA 2024/11/23-14:51:42.410095 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264615.sst): first pass time = 0.72's, 185.395'MB/sec 2024/11/23-14:51:43.231152 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264615.sst): old prealloc_size = 147639500, real_size = 40905104 2024/11/23-14:51:43.231291 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264615.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.76 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1017.01 avg-zval = 314.03 2024/11/23-14:51:43.231448 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264615.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:51:43.394222 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264615.sst): second pass time = 0.82's, 160.589'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 261.936'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1970.322'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.271'MB/sec write SST data time = 0.04's, 980.165'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 46708.770'MB/sec zip my value time = 0.82's, unzip length = 0.132'GB zip my value throughput = 160.589'MB/sec zip pipeline throughput = 160.589'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.76 avg-val = 1017.01 avg-zval = 314.03 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.76 avg-val = 1023.27 avg-zval = 314.03 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 115830 bytes = 434384 } seq expand size = 810810 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.8245 value = 3.2385 dict = 1.00 all = 3.1428 } Zip/UnZip{ index = 0.1465 value = 0.3088 dict = 1.00 all = 0.3182 } ---------------------------- total value len = 10.968009 GB avg = 0.202 KB (by entry num) total key len = 1.482239 GB avg = 0.027 KB total ukey len = 1.022729 GB avg = 0.015 KB total ukey num = 0.067621318 Billion total entry num = 0.054429047 Billion write speed all = 0.582027195 MB/sec (with seq num) write speed all = 0.561671497 MB/sec (without seq num) 2024/11/23-14:51:43.447757 139957685155392 [compaction_job.cc:1948] [default] [JOB 28440] Generated table #264615: 129429 keys, 42707141 bytes, temperature: kWarm 2024/11/23-14:51:43.447827 139957685155392 EVENT_LOG_v1 {"time_micros": 1732344703447791, "cf_name": "default", "job": 28440, "event": "table_file_creation", "file_number": 264615, "file_size": 42707141, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4041077626, "table_properties": {"data_size": 40644992, "index_size": 227584, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131630131, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344701, "oldest_key_time": 0, "file_creation_time": 1732344701, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264615, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:51:43.484968 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264616.sst) /dev/shm/ToplingTemp/Topling-81986-xuwwSz 2024/11/23-14:51:43.985351 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264616.sst): first pass time = 0.50's, 268.262'MB/sec 2024/11/23-14:51:44.796489 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264616.sst): old prealloc_size = 147639500, real_size = 40477568 2024/11/23-14:51:44.796621 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264616.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.73 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1017.00 avg-zval = 310.76 2024/11/23-14:51:44.796725 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264616.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:51:44.974680 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264616.sst): second pass time = 0.81's, 162.422'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 235.664'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1799.702'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.277'MB/sec write SST data time = 0.06's, 686.863'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 49269.708'MB/sec zip my value time = 0.81's, unzip length = 0.132'GB zip my value throughput = 162.422'MB/sec zip pipeline throughput = 162.422'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.73 avg-val = 1017.00 avg-zval = 310.76 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.73 avg-val = 1018.95 avg-zval = 310.76 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 36045 bytes = 135184 } seq expand size = 252315 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0420 GB } UnZip/Zip{ index = 6.9516 value = 3.2726 dict = 1.00 all = 3.1971 } Zip/UnZip{ index = 0.1439 value = 0.3056 dict = 1.00 all = 0.3128 } ---------------------------- total value len = 11.099891 GB avg = 0.203 KB (by entry num) total key len = 1.484827 GB avg = 0.027 KB total ukey len = 1.024282 GB avg = 0.015 KB total ukey num = 0.067750747 Billion total entry num = 0.054558476 Billion write speed all = 0.588269976 MB/sec (with seq num) write speed all = 0.567867382 MB/sec (without seq num) 2024/11/23-14:51:45.029441 139957685155392 [compaction_job.cc:1948] [default] [JOB 28440] Generated table #264616: 129429 keys, 41980554 bytes, temperature: kWarm 2024/11/23-14:51:45.029533 139957685155392 EVENT_LOG_v1 {"time_micros": 1732344705029485, "cf_name": "default", "job": 28440, "event": "table_file_creation", "file_number": 264616, "file_size": 41980554, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4041066613, "table_properties": {"data_size": 40221616, "index_size": 223424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629288, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344703, "oldest_key_time": 0, "file_creation_time": 1732344703, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264616, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:51:45.102117 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264617.sst) /dev/shm/ToplingTemp/Topling-81986-n7URhC 2024/11/23-14:51:45.816218 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264617.sst): first pass time = 0.71's, 187.970'MB/sec 2024/11/23-14:51:46.624008 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264617.sst): old prealloc_size = 147639500, real_size = 40583728 2024/11/23-14:51:46.624136 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264617.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.02 avg-zval = 311.60 2024/11/23-14:51:46.624225 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264617.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:51:46.787997 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264617.sst): second pass time = 0.81's, 163.198'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 256.474'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1961.855'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.268'MB/sec write SST data time = 0.04's, 971.609'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 42737.270'MB/sec zip my value time = 0.81's, unzip length = 0.132'GB zip my value throughput = 163.198'MB/sec zip pipeline throughput = 163.198'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.02 avg-zval = 311.60 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1018.46 avg-zval = 311.60 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 26510 bytes = 99424 } seq expand size = 185570 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0421 GB } UnZip/Zip{ index = 6.9935 value = 3.2639 dict = 1.00 all = 3.1918 } Zip/UnZip{ index = 0.1430 value = 0.3064 dict = 1.00 all = 0.3133 } ---------------------------- total value len = 11.231707 GB avg = 0.205 KB (by entry num) total key len = 1.487416 GB avg = 0.027 KB total ukey len = 1.025835 GB avg = 0.015 KB total ukey num = 0.067880174 Billion total entry num = 0.054687903 Billion write speed all = 0.594502286 MB/sec (with seq num) write speed all = 0.574053024 MB/sec (without seq num) 2024/11/23-14:51:46.839225 139957685155392 [compaction_job.cc:1948] [default] [JOB 28440] Generated table #264617: 129427 keys, 42051001 bytes, temperature: kWarm 2024/11/23-14:51:46.839289 139957685155392 EVENT_LOG_v1 {"time_micros": 1732344706839255, "cf_name": "default", "job": 28440, "event": "table_file_creation", "file_number": 264617, "file_size": 42051001, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4041077545, "table_properties": {"data_size": 40329120, "index_size": 222080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131630220, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344705, "oldest_key_time": 0, "file_creation_time": 1732344705, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264617, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:51:46.875427 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264618.sst) /dev/shm/ToplingTemp/Topling-81986-yQmxiB 2024/11/23-14:51:47.460368 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264618.sst): first pass time = 0.58's, 229.470'MB/sec 2024/11/23-14:51:48.271814 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264618.sst): old prealloc_size = 147639500, real_size = 40451840 2024/11/23-14:51:48.271934 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264618.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1017.00 avg-zval = 310.57 2024/11/23-14:51:48.272033 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264618.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:51:48.432808 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264618.sst): second pass time = 0.81's, 162.349'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 260.400'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2291.398'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.273'MB/sec write SST data time = 0.04's, 988.170'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 34794.046'MB/sec zip my value time = 0.81's, unzip length = 0.132'GB zip my value throughput = 162.349'MB/sec zip pipeline throughput = 162.349'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.00 avg-zval = 310.57 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1018.40 avg-zval = 310.57 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 25851 bytes = 96960 } seq expand size = 180957 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0419 GB } UnZip/Zip{ index = 6.9936 value = 3.2746 dict = 1.00 all = 3.2020 } Zip/UnZip{ index = 0.1430 value = 0.3054 dict = 1.00 all = 0.3123 } ---------------------------- total value len = 11.363517 GB avg = 0.207 KB (by entry num) total key len = 1.490004 GB avg = 0.027 KB total ukey len = 1.027388 GB avg = 0.015 KB total ukey num = 0.068009603 Billion total entry num = 0.054817332 Billion write speed all = 0.600738011 MB/sec (with seq num) write speed all = 0.580241928 MB/sec (without seq num) 2024/11/23-14:51:48.482775 139957685155392 [compaction_job.cc:1948] [default] [JOB 28440] Generated table #264618: 129429 keys, 41916548 bytes, temperature: kWarm 2024/11/23-14:51:48.482845 139957685155392 EVENT_LOG_v1 {"time_micros": 1732344708482809, "cf_name": "default", "job": 28440, "event": "table_file_creation", "file_number": 264618, "file_size": 41916548, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4041069108, "table_properties": {"data_size": 40197232, "index_size": 222080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629244, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344706, "oldest_key_time": 0, "file_creation_time": 1732344706, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264618, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:51:48.522184 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264619.sst) /dev/shm/ToplingTemp/Topling-81986-cAMjzz 2024/11/23-14:51:49.065659 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264619.sst): first pass time = 0.54's, 246.989'MB/sec 2024/11/23-14:51:49.835253 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264619.sst): old prealloc_size = 147639500, real_size = 40692720 2024/11/23-14:51:49.835371 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264619.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1017.01 avg-zval = 312.44 2024/11/23-14:51:49.835487 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264619.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:51:49.993074 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264619.sst): second pass time = 0.77's, 171.201'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 268.197'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2286.326'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.280'MB/sec write SST data time = 0.04's, 1006.754'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 50621.621'MB/sec zip my value time = 0.77's, unzip length = 0.132'GB zip my value throughput = 171.201'MB/sec zip pipeline throughput = 171.201'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.01 avg-zval = 312.44 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1020.77 avg-zval = 312.44 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 69506 bytes = 260656 } seq expand size = 486542 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0423 GB } UnZip/Zip{ index = 6.9936 value = 3.2551 dict = 1.00 all = 3.1714 } Zip/UnZip{ index = 0.1430 value = 0.3072 dict = 1.00 all = 0.3153 } ---------------------------- total value len = 11.495633 GB avg = 0.209 KB (by entry num) total key len = 1.492593 GB avg = 0.027 KB total ukey len = 1.028941 GB avg = 0.015 KB total ukey num = 0.068139031 Billion total entry num = 0.054946760 Billion write speed all = 0.606989468 MB/sec (with seq num) write speed all = 0.586446491 MB/sec (without seq num) 2024/11/23-14:51:50.044262 139957685155392 [compaction_job.cc:1948] [default] [JOB 28440] Generated table #264619: 129428 keys, 42321226 bytes, temperature: kWarm 2024/11/23-14:51:50.044330 139957685155392 EVENT_LOG_v1 {"time_micros": 1732344710044294, "cf_name": "default", "job": 28440, "event": "table_file_creation", "file_number": 264619, "file_size": 42321226, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4041082576, "table_properties": {"data_size": 40438112, "index_size": 222080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629512, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344708, "oldest_key_time": 0, "file_creation_time": 1732344708, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264619, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:51:50.084222 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264620.sst) /dev/shm/ToplingTemp/Topling-81986-B07coD 2024/11/23-14:51:50.093273 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264620.sst): first pass time = 0.01's, 449.511'MB/sec 2024/11/23-14:51:50.127379 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264620.sst): old prealloc_size = 147639500, real_size = 1274152 2024/11/23-14:51:50.127424 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264620.sst): index type = UintIndex_AllOne , store type = DictZipBlobStore usrkeys = 3910 prefix = 4 raw-key = 0.0000 GB zip-key = 0.0000 GB avg-key = 8.00 avg-zkey = 0.04 raw-val = 0.0040 GB zip-val = 0.0013 GB avg-val =1016.89 avg-zval = 325.53 2024/11/23-14:51:50.134559 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00/264620.sst): second pass time = 0.03's, 118.449'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.01's, 185.556'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 6.721'MB/sec (index lex order gen) rebuild zvType time = 0.01's, 0.175'MB/sec write SST data time = 0.00's, 861.961'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.043'MB, throughput = 1980.099'MB/sec zip my value time = 0.03's, unzip length = 0.004'GB zip my value throughput = 118.449'MB/sec zip pipeline throughput = 118.449'MB/sec entries = 3910 avg-key = 20.00 avg-zkey = 0.05 avg-val = 1016.89 avg-zval = 325.53 usrkeys = 3910 avg-key = 12.00 avg-zkey = 0.05 avg-val = 1021.75 avg-zval = 325.53 TagRS{ kinds = RS_Key0_TagN bytes = 720 } TagArray{ size = 2712 bytes = 10192 } seq expand size = 18984 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0000 GB value = 0.0040 GB dict = 0.04 MB all = 0.0041 GB } __ZipSize{ index = 0.0000 GB value = 0.0013 GB dict = 0.04 MB all = 0.0013 GB } UnZip/Zip{ index = 244.3750 value = 3.1238 dict = 1.00 all = 3.0533 } Zip/UnZip{ index = 0.0041 value = 0.3201 dict = 1.00 all = 0.3275 } ---------------------------- total value len = 11.499628 GB avg = 0.209 KB (by entry num) total key len = 1.492671 GB avg = 0.027 KB total ukey len = 1.028988 GB avg = 0.015 KB total ukey num = 0.068142941 Billion total entry num = 0.054950670 Billion write speed all = 0.607175811 MB/sec (with seq num) write speed all = 0.586631507 MB/sec (without seq num) 2024/11/23-14:51:50.142592 139957685155392 [compaction_job.cc:1948] [default] [JOB 28440] Generated table #264620: 3910 keys, 1327803 bytes, temperature: kWarm 2024/11/23-14:51:50.142650 139957685155392 EVENT_LOG_v1 {"time_micros": 1732344710142621, "cf_name": "default", "job": 28440, "event": "table_file_creation", "file_number": 264620, "file_size": 1327803, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4041082313, "table_properties": {"data_size": 1272816, "index_size": 192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 78200, "raw_average_key_size": 20, "raw_value_size": 3976039, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 3910, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-UintIndex_AllOne-DictZipBlobStore", "creation_time": 1732344710, "oldest_key_time": 0, "file_creation_time": 1732344710, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264620, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:51:50.151710 139957685155392 [compaction_job.cc:703] job-28440: subcompact[0], size: 0.850943 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28440/att-00 [264600,264601,264602,264603,264604,264605,264606,264607,264608,264609,264610,264611,264612,264613,264614,264615,264616,264617,264618,264619,264620] 2024/11/23-14:51:50.286172 139957685155392 (Original Log Time 2024/11/23-14:51:11.209397) EVENT_LOG_v1 {"time_micros": 1732344671209331, "job": 28440, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [264579], "files_L3": [264497, 264498, 264499, 264500, 264501, 264502, 264503, 264504, 264505, 264506, 264507, 264508, 264509, 264510, 264511, 264512, 264513, 264514, 264515, 264596], "score": 0.9902, "input_data_size": 882271833, "oldest_snapshot_seqno": 4041182030} 2024/11/23-14:51:50.286175 139957685155392 (Original Log Time 2024/11/23-14:51:50.284208) [compaction_job.cc:2022] [default] [JOB 28440] Compacted 1@2 + 20@3 files to L3 => 850942538 bytes 2024/11/23-14:51:50.286176 139957685155392 (Original Log Time 2024/11/23-14:51:50.286108) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 21 0 0 0] max score 0.03, MB/sec: 22.7 rd, 21.9 wr, level 3, files in(1, 20) out(21 +0 blob) MB in(31.2, 810.2 +0.0 blob) out(811.5 +0.0 blob), read-write-amplify(53.0) write-amplify(26.0) OK, records in: 2620504, records dropped: 27998 output_compression: Snappy 2024/11/23-14:51:50.286180 139957685155392 (Original Log Time 2024/11/23-14:51:50.286147) EVENT_LOG_v1 {"time_micros": 1732344710286128, "cf": "default", "job": 28440, "event": "compaction_finished", "compaction_time_micros": 38942315, "compaction_time_cpu_micros": 7833551, "output_level": 3, "num_output_files": 21, "total_output_size": 850942538, "num_input_records": 2620504, "num_output_records": 2592506, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 593629085, "file_range_sync_nanos": 0, "file_fsync_nanos": 14701, "file_prepare_write_nanos": 52925, "lsm_state": [0, 0, 0, 21, 0, 0, 0]} 2024/11/23-14:51:50.423313 139957685155392 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732344671,"executesMs":39207,"compactionJobId":28440,"attempt":0,"compactionInputRawBytes":2717451841,"compactionInputZipBytes":882271833,"compactionOutputRawBytes":0,"compactionOutputZipBytes":850942538,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-8383dd70e9c1073ba4c5123d11a34ce3-393ce58cc1e1fe86-00"}}, response = 2024/11/23-14:51:50.423389 139957685155392 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28440/att-00: olev 3, work 39.207 s, result 7.743 ms, install 1.986 ms, input{raw 2.531 GiB zip 841.400 MiB}