2024/11/23-21:31:39.481683 139957643191872 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00: fork to child time = 0.007510 sec 2024/11/23-21:31:39.482454 139957643191872 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00 2024/11/23-21:31:39.490129 139957643191872 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00 2024/11/23-21:31:39.491052 139957643191872 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-21:31:39.491317 139957643191872 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-21:31:39.544273 139957643191872 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-36224/att-00/MANIFEST-206347 2024/11/23-21:31:39.576293 139957643191872 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-36224/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 281669, last_sequence is 4274435510, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 281433 2024/11/23-21:31:39.576299 139957643191872 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-21:31:39.576568 139957643191872 [version_set.cc:5721] Creating manifest 281669 2024/11/23-21:31:39.610112 139957643191872 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-21:31:39.629262 139957643191872 [compaction_job.cc:2437] [default] [JOB 36224] Compacting 11@3 + 28@4 files to L4, score -1.00, subcompactions 1 : 1 2024/11/23-21:31:39.629284 139957643191872 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 27111 Base level 3, inputs: [281664(40MB) 281665(40MB) 281666(40MB) 281667(94KB) 281599(40MB) 281600(86KB) 281588(15MB) 281593(25MB) 281595(3306KB) 281601(29MB) 281602(1682KB)], [281657(79MB) 281658(79MB) 281659(79MB) 281660(79MB) 281661(31MB) 209784(40MB) 209785(39MB) 209786(39MB) 209787(39MB) 210856(39MB) 210858(39MB) 210857(39MB) 210863(39MB) 211883(39MB) 211884(39MB) 211885(39MB) 211886(39MB) 276543(80MB) 276544(30MB) 212835(39MB) 212832(10MB) 261641(50MB) 261642(16MB) 260306(24MB) 280232(10MB) 270877(60MB) 270878(30MB) 265852(30MB)] 2024/11/23-21:31:39.637074 139957643191872 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-21:31:39.637100 139957643191872 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-21:31:39.637119 139957643191872 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-21:31:39.637125 139957643191872 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-21:31:39.637135 139957643191872 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-21:31:41.058795 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281670.sst) /dev/shm/ToplingTemp/Topling-194742-bMVA8F 2024/11/23-21:31:42.171302 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281670.sst): first pass time = 1.11's, 241.298'MB/sec 2024/11/23-21:31:43.676236 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281670.sst): old prealloc_size = 295279001, real_size = 80480256 2024/11/23-21:31:43.676431 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281670.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 258855 prefix = 4 raw-key = 0.0021 GB zip-key = 0.0004 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.2633 GB zip-val = 0.0800 GB avg-val =1017.01 avg-zval = 308.97 2024/11/23-21:31:43.676525 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281670.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0006 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:31:43.994540 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281670.sst): second pass time = 1.50's, 175.095'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.32's, 261.225'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2519.927'MB/sec (index lex order gen) rebuild zvType time = 0.23's, 0.277'MB/sec write SST data time = 0.08's, 987.225'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 2.689'MB, throughput = 80476.146'MB/sec zip my value time = 1.50's, unzip length = 0.263'GB zip my value throughput = 175.095'MB/sec zip pipeline throughput = 175.095'MB/sec entries = 258855 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.01 avg-zval = 308.97 usrkeys = 258855 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.01 avg-zval = 308.97 TagRS{ kinds = RS_Key0_TagN bytes = 40560 } TagArray{ size = 26 bytes = 80 } seq expand size = 182 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0031 GB value = 0.2633 GB dict = 2.69 MB all = 0.2684 GB } __ZipSize{ index = 0.0004 GB value = 0.0800 GB dict = 2.69 MB all = 0.0831 GB } UnZip/Zip{ index = 7.1229 value = 3.2916 dict = 1.00 all = 3.2284 } Zip/UnZip{ index = 0.1404 value = 0.3038 dict = 1.00 all = 0.3097 } ---------------------------- total value len = 9.111879 GB avg = 0.173 KB (by entry num) total key len = 1.445998 GB avg = 0.027 KB total ukey len = 1.000984 GB avg = 0.015 KB total ukey num = 0.065809290 Billion total entry num = 0.052617019 Billion write speed all = 0.232594467 MB/sec (with seq num) write speed all = 0.223321066 MB/sec (without seq num) 2024/11/23-21:31:44.090945 139957643191872 [compaction_job.cc:1948] [default] [JOB 36224] Generated table #281670: 258855 keys, 83147301 bytes, temperature: kWarm 2024/11/23-21:31:44.091069 139957643191872 EVENT_LOG_v1 {"time_micros": 1732368704091002, "cf_name": "default", "job": 36224, "event": "table_file_creation", "file_number": 281670, "file_size": 83147301, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4273199046, "table_properties": {"data_size": 79979248, "index_size": 436096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 5177100, "raw_average_key_size": 20, "raw_value_size": 263258689, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 258855, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732368701, "oldest_key_time": 0, "file_creation_time": 1732368701, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 281670, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:31:44.151865 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281671.sst) /dev/shm/ToplingTemp/Topling-194742-VoGX0E 2024/11/23-21:31:45.374604 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281671.sst): first pass time = 1.22's, 219.543'MB/sec 2024/11/23-21:31:46.776767 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281671.sst): old prealloc_size = 295279001, real_size = 80480528 2024/11/23-21:31:46.776956 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281671.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 258858 prefix = 4 raw-key = 0.0021 GB zip-key = 0.0004 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.2633 GB zip-val = 0.0800 GB avg-val =1017.00 avg-zval = 308.97 2024/11/23-21:31:46.777070 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281671.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0006 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:31:47.097576 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281671.sst): second pass time = 1.40's, 187.942'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.32's, 259.199'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2628.494'MB/sec (index lex order gen) rebuild zvType time = 0.23's, 0.279'MB/sec write SST data time = 0.09's, 937.429'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 2.689'MB, throughput = 78173.649'MB/sec zip my value time = 1.40's, unzip length = 0.263'GB zip my value throughput = 187.942'MB/sec zip pipeline throughput = 187.942'MB/sec entries = 258858 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 308.97 usrkeys = 258858 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 308.97 TagRS{ kinds = RS_Key0_TagN bytes = 40560 } TagArray{ size = 33 bytes = 96 } seq expand size = 231 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0031 GB value = 0.2633 GB dict = 2.69 MB all = 0.2684 GB } __ZipSize{ index = 0.0004 GB value = 0.0800 GB dict = 2.69 MB all = 0.0831 GB } UnZip/Zip{ index = 7.1230 value = 3.2916 dict = 1.00 all = 3.2284 } Zip/UnZip{ index = 0.1404 value = 0.3038 dict = 1.00 all = 0.3097 } ---------------------------- total value len = 9.375137 GB avg = 0.177 KB (by entry num) total key len = 1.451175 GB avg = 0.027 KB total ukey len = 1.004091 GB avg = 0.015 KB total ukey num = 0.066068148 Billion total entry num = 0.052875877 Billion write speed all = 0.238491917 MB/sec (with seq num) write speed all = 0.229173532 MB/sec (without seq num) 2024/11/23-21:31:47.191204 139957643191872 [compaction_job.cc:1948] [default] [JOB 36224] Generated table #281671: 258858 keys, 83147627 bytes, temperature: kWarm 2024/11/23-21:31:47.191314 139957643191872 EVENT_LOG_v1 {"time_micros": 1732368707191278, "cf_name": "default", "job": 36224, "event": "table_file_creation", "file_number": 281671, "file_size": 83147627, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4273174519, "table_properties": {"data_size": 79979520, "index_size": 436096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 5177160, "raw_average_key_size": 20, "raw_value_size": 263258433, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 258858, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732368704, "oldest_key_time": 0, "file_creation_time": 1732368704, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 281671, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:31:47.246533 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281672.sst) /dev/shm/ToplingTemp/Topling-194742-SDzrHG 2024/11/23-21:31:48.616919 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281672.sst): first pass time = 1.37's, 195.889'MB/sec 2024/11/23-21:31:50.031454 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281672.sst): old prealloc_size = 295279001, real_size = 80481136 2024/11/23-21:31:50.031638 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281672.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 258853 prefix = 4 raw-key = 0.0021 GB zip-key = 0.0004 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.2633 GB zip-val = 0.0800 GB avg-val =1017.02 avg-zval = 308.98 2024/11/23-21:31:50.031714 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281672.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0006 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:31:50.351675 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281672.sst): second pass time = 1.41's, 186.311'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.32's, 259.679'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2727.697'MB/sec (index lex order gen) rebuild zvType time = 0.23's, 0.278'MB/sec write SST data time = 0.09's, 952.750'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 2.690'MB, throughput = 78607.651'MB/sec zip my value time = 1.41's, unzip length = 0.263'GB zip my value throughput = 186.311'MB/sec zip pipeline throughput = 186.311'MB/sec entries = 258853 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.02 avg-zval = 308.98 usrkeys = 258853 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.02 avg-zval = 308.98 TagRS{ kinds = RS_Key0_TagN bytes = 40560 } TagArray{ size = 31 bytes = 96 } seq expand size = 217 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0031 GB value = 0.2633 GB dict = 2.69 MB all = 0.2684 GB } __ZipSize{ index = 0.0004 GB value = 0.0800 GB dict = 2.69 MB all = 0.0831 GB } UnZip/Zip{ index = 7.1228 value = 3.2916 dict = 1.00 all = 3.2284 } Zip/UnZip{ index = 0.1404 value = 0.3038 dict = 1.00 all = 0.3098 } ---------------------------- total value len = 9.638397 GB avg = 0.181 KB (by entry num) total key len = 1.456352 GB avg = 0.027 KB total ukey len = 1.007197 GB avg = 0.015 KB total ukey num = 0.066327001 Billion total entry num = 0.053134730 Billion write speed all = 0.244387761 MB/sec (with seq num) write speed all = 0.235024429 MB/sec (without seq num) 2024/11/23-21:31:50.445230 139957643191872 [compaction_job.cc:1948] [default] [JOB 36224] Generated table #281672: 258853 keys, 83149046 bytes, temperature: kWarm 2024/11/23-21:31:50.445299 139957643191872 EVENT_LOG_v1 {"time_micros": 1732368710445263, "cf_name": "default", "job": 36224, "event": "table_file_creation", "file_number": 281672, "file_size": 83149046, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4273175173, "table_properties": {"data_size": 79980128, "index_size": 436096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 5177060, "raw_average_key_size": 20, "raw_value_size": 263259121, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 258853, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732368707, "oldest_key_time": 0, "file_creation_time": 1732368707, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 281672, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:31:50.501932 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281673.sst) /dev/shm/ToplingTemp/Topling-194742-Y3CnwE 2024/11/23-21:31:51.188030 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281673.sst): first pass time = 0.69's, 391.275'MB/sec 2024/11/23-21:31:52.654527 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281673.sst): old prealloc_size = 295279001, real_size = 80477904 2024/11/23-21:31:52.654707 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281673.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 258863 prefix = 4 raw-key = 0.0021 GB zip-key = 0.0004 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.2633 GB zip-val = 0.0800 GB avg-val =1016.98 avg-zval = 308.95 2024/11/23-21:31:52.654786 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281673.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0006 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:31:52.977248 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281673.sst): second pass time = 1.47's, 179.687'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.32's, 257.659'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2780.249'MB/sec (index lex order gen) rebuild zvType time = 0.23's, 0.277'MB/sec write SST data time = 0.09's, 936.366'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 2.689'MB, throughput = 86341.254'MB/sec zip my value time = 1.47's, unzip length = 0.263'GB zip my value throughput = 179.687'MB/sec zip pipeline throughput = 179.687'MB/sec entries = 258863 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1016.98 avg-zval = 308.95 usrkeys = 258863 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1016.98 avg-zval = 308.95 TagRS{ kinds = RS_Key0_TagN bytes = 40560 } TagArray{ size = 35 bytes = 112 } seq expand size = 245 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0031 GB value = 0.2633 GB dict = 2.69 MB all = 0.2684 GB } __ZipSize{ index = 0.0004 GB value = 0.0800 GB dict = 2.69 MB all = 0.0831 GB } UnZip/Zip{ index = 7.1231 value = 3.2917 dict = 1.00 all = 3.2285 } Zip/UnZip{ index = 0.1404 value = 0.3038 dict = 1.00 all = 0.3097 } ---------------------------- total value len = 9.901656 GB avg = 0.185 KB (by entry num) total key len = 1.461530 GB avg = 0.027 KB total ukey len = 1.010303 GB avg = 0.015 KB total ukey num = 0.066585864 Billion total entry num = 0.053393593 Billion write speed all = 0.250286220 MB/sec (with seq num) write speed all = 0.240877816 MB/sec (without seq num) 2024/11/23-21:31:53.072399 139957643191872 [compaction_job.cc:1948] [default] [JOB 36224] Generated table #281673: 258863 keys, 83144817 bytes, temperature: kWarm 2024/11/23-21:31:53.072472 139957643191872 EVENT_LOG_v1 {"time_micros": 1732368713072433, "cf_name": "default", "job": 36224, "event": "table_file_creation", "file_number": 281673, "file_size": 83144817, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4273192763, "table_properties": {"data_size": 79976896, "index_size": 436096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 5177260, "raw_average_key_size": 20, "raw_value_size": 263258696, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 258863, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732368710, "oldest_key_time": 0, "file_creation_time": 1732368710, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 281673, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:31:53.127716 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281674.sst) /dev/shm/ToplingTemp/Topling-194742-8wAQFF 2024/11/23-21:31:54.014417 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281674.sst): first pass time = 0.89's, 302.751'MB/sec 2024/11/23-21:31:55.535290 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281674.sst): old prealloc_size = 295279001, real_size = 80863168 2024/11/23-21:31:55.535479 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281674.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 258856 prefix = 4 raw-key = 0.0021 GB zip-key = 0.0004 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.2633 GB zip-val = 0.0804 GB avg-val =1017.01 avg-zval = 310.45 2024/11/23-21:31:55.535591 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281674.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0006 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:31:55.860951 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281674.sst): second pass time = 1.52's, 173.247'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.33's, 256.629'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2623.434'MB/sec (index lex order gen) rebuild zvType time = 0.23's, 0.276'MB/sec write SST data time = 0.09's, 922.512'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 2.689'MB, throughput = 82471.552'MB/sec zip my value time = 1.52's, unzip length = 0.263'GB zip my value throughput = 173.247'MB/sec zip pipeline throughput = 173.247'MB/sec entries = 258856 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.01 avg-zval = 310.45 usrkeys = 258856 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.29 avg-zval = 310.45 TagRS{ kinds = RS_Key0_TagN bytes = 40560 } TagArray{ size = 10492 bytes = 39360 } seq expand size = 73444 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0031 GB value = 0.2633 GB dict = 2.69 MB all = 0.2684 GB } __ZipSize{ index = 0.0004 GB value = 0.0804 GB dict = 2.69 MB all = 0.0836 GB } UnZip/Zip{ index = 7.1229 value = 3.2759 dict = 1.00 all = 3.2121 } Zip/UnZip{ index = 0.1404 value = 0.3053 dict = 1.00 all = 0.3113 } ---------------------------- total value len = 10.164987 GB avg = 0.189 KB (by entry num) total key len = 1.466707 GB avg = 0.027 KB total ukey len = 1.013409 GB avg = 0.015 KB total ukey num = 0.066844720 Billion total entry num = 0.053652449 Billion write speed all = 0.256184142 MB/sec (with seq num) write speed all = 0.246730726 MB/sec (without seq num) 2024/11/23-21:31:55.956240 139957643191872 [compaction_job.cc:1948] [default] [JOB 36224] Generated table #281674: 258856 keys, 83569396 bytes, temperature: kWarm 2024/11/23-21:31:55.956308 139957643191872 EVENT_LOG_v1 {"time_micros": 1732368715956273, "cf_name": "default", "job": 36224, "event": "table_file_creation", "file_number": 281674, "file_size": 83569396, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4273163242, "table_properties": {"data_size": 80362160, "index_size": 436096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 5177120, "raw_average_key_size": 20, "raw_value_size": 263258357, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 258856, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732368713, "oldest_key_time": 0, "file_creation_time": 1732368713, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 281674, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:31:56.012328 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281675.sst) /dev/shm/ToplingTemp/Topling-194742-KhzREE 2024/11/23-21:31:57.334907 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281675.sst): first pass time = 1.32's, 202.971'MB/sec 2024/11/23-21:31:58.802497 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281675.sst): old prealloc_size = 295279001, real_size = 80527200 2024/11/23-21:31:58.802680 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281675.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 258859 prefix = 4 raw-key = 0.0021 GB zip-key = 0.0004 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.2633 GB zip-val = 0.0800 GB avg-val =1017.00 avg-zval = 309.15 2024/11/23-21:31:58.802776 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281675.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0006 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:31:59.128886 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281675.sst): second pass time = 1.47's, 179.561'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.33's, 255.083'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2697.829'MB/sec (index lex order gen) rebuild zvType time = 0.23's, 0.276'MB/sec write SST data time = 0.09's, 907.331'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 2.690'MB, throughput = 83221.332'MB/sec zip my value time = 1.47's, unzip length = 0.263'GB zip my value throughput = 179.561'MB/sec zip pipeline throughput = 179.561'MB/sec entries = 258859 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 309.15 usrkeys = 258859 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.40 avg-zval = 309.15 TagRS{ kinds = RS_Key0_TagN bytes = 40560 } TagArray{ size = 14833 bytes = 55632 } seq expand size = 103831 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0031 GB value = 0.2633 GB dict = 2.69 MB all = 0.2684 GB } __ZipSize{ index = 0.0004 GB value = 0.0800 GB dict = 2.69 MB all = 0.0833 GB } UnZip/Zip{ index = 7.1230 value = 3.2897 dict = 1.00 all = 3.2244 } Zip/UnZip{ index = 0.1404 value = 0.3040 dict = 1.00 all = 0.3101 } ---------------------------- total value len = 10.428350 GB avg = 0.193 KB (by entry num) total key len = 1.471884 GB avg = 0.027 KB total ukey len = 1.016516 GB avg = 0.015 KB total ukey num = 0.067103579 Billion total entry num = 0.053911308 Billion write speed all = 0.262079770 MB/sec (with seq num) write speed all = 0.252581427 MB/sec (without seq num) 2024/11/23-21:31:59.223011 139957643191872 [compaction_job.cc:1948] [default] [JOB 36224] Generated table #281675: 258859 keys, 83250443 bytes, temperature: kWarm 2024/11/23-21:31:59.223082 139957643191872 EVENT_LOG_v1 {"time_micros": 1732368719223045, "cf_name": "default", "job": 36224, "event": "table_file_creation", "file_number": 281675, "file_size": 83250443, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4273180044, "table_properties": {"data_size": 80026192, "index_size": 436096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 5177180, "raw_average_key_size": 20, "raw_value_size": 263258439, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 258859, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732368716, "oldest_key_time": 0, "file_creation_time": 1732368716, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 281675, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:31:59.283485 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281676.sst) /dev/shm/ToplingTemp/Topling-194742-CGdkAF 2024/11/23-21:32:01.190852 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281676.sst): first pass time = 1.91's, 140.739'MB/sec 2024/11/23-21:32:02.673842 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281676.sst): old prealloc_size = 295279001, real_size = 80481120 2024/11/23-21:32:02.674050 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281676.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 258855 prefix = 4 raw-key = 0.0021 GB zip-key = 0.0004 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.2633 GB zip-val = 0.0800 GB avg-val =1017.01 avg-zval = 308.98 2024/11/23-21:32:02.674145 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281676.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0006 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:32:02.999897 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281676.sst): second pass time = 1.48's, 177.697'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.33's, 255.198'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2354.692'MB/sec (index lex order gen) rebuild zvType time = 0.23's, 0.277'MB/sec write SST data time = 0.09's, 898.263'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 2.689'MB, throughput = 75753.387'MB/sec zip my value time = 1.48's, unzip length = 0.263'GB zip my value throughput = 177.697'MB/sec zip pipeline throughput = 177.697'MB/sec entries = 258855 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.01 avg-zval = 308.98 usrkeys = 258855 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.41 avg-zval = 308.98 TagRS{ kinds = RS_Key0_TagN bytes = 40560 } TagArray{ size = 14577 bytes = 54672 } seq expand size = 102039 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0031 GB value = 0.2633 GB dict = 2.69 MB all = 0.2684 GB } __ZipSize{ index = 0.0004 GB value = 0.0800 GB dict = 2.69 MB all = 0.0832 GB } UnZip/Zip{ index = 7.1229 value = 3.2916 dict = 1.00 all = 3.2263 } Zip/UnZip{ index = 0.1404 value = 0.3038 dict = 1.00 all = 0.3100 } ---------------------------- total value len = 10.691710 GB avg = 0.197 KB (by entry num) total key len = 1.477061 GB avg = 0.027 KB total ukey len = 1.019622 GB avg = 0.015 KB total ukey num = 0.067362434 Billion total entry num = 0.054170163 Billion write speed all = 0.267970954 MB/sec (with seq num) write speed all = 0.258427818 MB/sec (without seq num) 2024/11/23-21:32:03.093669 139957643191872 [compaction_job.cc:1948] [default] [JOB 36224] Generated table #281676: 258855 keys, 83203177 bytes, temperature: kWarm 2024/11/23-21:32:03.093739 139957643191872 EVENT_LOG_v1 {"time_micros": 1732368723093703, "cf_name": "default", "job": 36224, "event": "table_file_creation", "file_number": 281676, "file_size": 83203177, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4273101722, "table_properties": {"data_size": 79980112, "index_size": 436096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 5177100, "raw_average_key_size": 20, "raw_value_size": 263258664, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 258855, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732368719, "oldest_key_time": 0, "file_creation_time": 1732368719, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 281676, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:32:03.150889 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281677.sst) /dev/shm/ToplingTemp/Topling-194742-3nldBE 2024/11/23-21:32:04.034363 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281677.sst): first pass time = 0.88's, 303.855'MB/sec 2024/11/23-21:32:05.496744 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281677.sst): old prealloc_size = 295279001, real_size = 80477760 2024/11/23-21:32:05.496927 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281677.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 258863 prefix = 4 raw-key = 0.0021 GB zip-key = 0.0004 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.2633 GB zip-val = 0.0800 GB avg-val =1016.98 avg-zval = 308.95 2024/11/23-21:32:05.497041 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281677.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0006 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:32:05.817100 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281677.sst): second pass time = 1.46's, 180.194'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.32's, 259.726'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2724.068'MB/sec (index lex order gen) rebuild zvType time = 0.23's, 0.279'MB/sec write SST data time = 0.09's, 941.032'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 2.689'MB, throughput = 80377.429'MB/sec zip my value time = 1.46's, unzip length = 0.263'GB zip my value throughput = 180.194'MB/sec zip pipeline throughput = 180.194'MB/sec entries = 258863 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1016.98 avg-zval = 308.95 usrkeys = 258863 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.38 avg-zval = 308.95 TagRS{ kinds = RS_Key0_TagN bytes = 40560 } TagArray{ size = 14552 bytes = 54592 } seq expand size = 101864 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0031 GB value = 0.2633 GB dict = 2.69 MB all = 0.2684 GB } __ZipSize{ index = 0.0004 GB value = 0.0800 GB dict = 2.69 MB all = 0.0832 GB } UnZip/Zip{ index = 7.1231 value = 3.2917 dict = 1.00 all = 3.2264 } Zip/UnZip{ index = 0.1404 value = 0.3038 dict = 1.00 all = 0.3099 } ---------------------------- total value len = 10.955071 GB avg = 0.201 KB (by entry num) total key len = 1.482238 GB avg = 0.027 KB total ukey len = 1.022728 GB avg = 0.015 KB total ukey num = 0.067621297 Billion total entry num = 0.054429026 Billion write speed all = 0.273867494 MB/sec (with seq num) write speed all = 0.264279349 MB/sec (without seq num) 2024/11/23-21:32:05.912634 139957643191872 [compaction_job.cc:1948] [default] [JOB 36224] Generated table #281677: 258863 keys, 83199025 bytes, temperature: kWarm 2024/11/23-21:32:05.912704 139957643191872 EVENT_LOG_v1 {"time_micros": 1732368725912669, "cf_name": "default", "job": 36224, "event": "table_file_creation", "file_number": 281677, "file_size": 83199025, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4273179635, "table_properties": {"data_size": 79976752, "index_size": 436096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 5177260, "raw_average_key_size": 20, "raw_value_size": 263258952, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 258863, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732368723, "oldest_key_time": 0, "file_creation_time": 1732368723, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 281677, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:32:05.970400 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281678.sst) /dev/shm/ToplingTemp/Topling-194742-RZJLyE 2024/11/23-21:32:07.357598 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281678.sst): first pass time = 1.39's, 193.516'MB/sec 2024/11/23-21:32:08.843006 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281678.sst): old prealloc_size = 295279001, real_size = 80476384 2024/11/23-21:32:08.843191 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281678.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 258860 prefix = 4 raw-key = 0.0021 GB zip-key = 0.0004 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.2633 GB zip-val = 0.0800 GB avg-val =1016.99 avg-zval = 308.95 2024/11/23-21:32:08.843291 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281678.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0006 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:32:09.168109 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281678.sst): second pass time = 1.48's, 177.390'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.33's, 255.915'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2534.926'MB/sec (index lex order gen) rebuild zvType time = 0.23's, 0.278'MB/sec write SST data time = 0.09's, 905.707'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 2.689'MB, throughput = 88581.925'MB/sec zip my value time = 1.48's, unzip length = 0.263'GB zip my value throughput = 177.390'MB/sec zip pipeline throughput = 177.390'MB/sec entries = 258860 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1016.99 avg-zval = 308.95 usrkeys = 258860 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.39 avg-zval = 308.95 TagRS{ kinds = RS_Key0_TagN bytes = 40560 } TagArray{ size = 14650 bytes = 54960 } seq expand size = 102550 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0031 GB value = 0.2633 GB dict = 2.69 MB all = 0.2684 GB } __ZipSize{ index = 0.0004 GB value = 0.0800 GB dict = 2.69 MB all = 0.0832 GB } UnZip/Zip{ index = 7.1230 value = 3.2917 dict = 1.00 all = 3.2265 } Zip/UnZip{ index = 0.1404 value = 0.3038 dict = 1.00 all = 0.3099 } ---------------------------- total value len = 11.218432 GB avg = 0.205 KB (by entry num) total key len = 1.487416 GB avg = 0.027 KB total ukey len = 1.025835 GB avg = 0.015 KB total ukey num = 0.067880157 Billion total entry num = 0.054687886 Billion write speed all = 0.279760024 MB/sec (with seq num) write speed all = 0.270126989 MB/sec (without seq num) 2024/11/23-21:32:09.266192 139957643191872 [compaction_job.cc:1948] [default] [JOB 36224] Generated table #281678: 258860 keys, 83197926 bytes, temperature: kWarm 2024/11/23-21:32:09.266267 139957643191872 EVENT_LOG_v1 {"time_micros": 1732368729266233, "cf_name": "default", "job": 36224, "event": "table_file_creation", "file_number": 281678, "file_size": 83197926, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4273142836, "table_properties": {"data_size": 79975376, "index_size": 436096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 5177200, "raw_average_key_size": 20, "raw_value_size": 263258730, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 258860, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732368725, "oldest_key_time": 0, "file_creation_time": 1732368725, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 281678, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:32:09.324100 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281679.sst) /dev/shm/ToplingTemp/Topling-194742-2C10LH 2024/11/23-21:32:11.267717 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281679.sst): first pass time = 1.94's, 138.114'MB/sec 2024/11/23-21:32:12.779480 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281679.sst): old prealloc_size = 295279001, real_size = 80476128 2024/11/23-21:32:12.779678 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281679.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 258859 prefix = 4 raw-key = 0.0021 GB zip-key = 0.0004 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.2633 GB zip-val = 0.0800 GB avg-val =1017.00 avg-zval = 308.95 2024/11/23-21:32:12.779771 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281679.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0006 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:32:13.104653 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281679.sst): second pass time = 1.51's, 174.292'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.33's, 255.875'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2460.538'MB/sec (index lex order gen) rebuild zvType time = 0.23's, 0.277'MB/sec write SST data time = 0.09's, 913.795'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 2.689'MB, throughput = 83362.227'MB/sec zip my value time = 1.51's, unzip length = 0.263'GB zip my value throughput = 174.292'MB/sec zip pipeline throughput = 174.292'MB/sec entries = 258859 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 308.95 usrkeys = 258859 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.38 avg-zval = 308.95 TagRS{ kinds = RS_Key0_TagN bytes = 40560 } TagArray{ size = 14337 bytes = 53776 } seq expand size = 100359 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0031 GB value = 0.2633 GB dict = 2.69 MB all = 0.2684 GB } __ZipSize{ index = 0.0004 GB value = 0.0800 GB dict = 2.69 MB all = 0.0832 GB } UnZip/Zip{ index = 7.1230 value = 3.2918 dict = 1.00 all = 3.2265 } Zip/UnZip{ index = 0.1404 value = 0.3038 dict = 1.00 all = 0.3099 } ---------------------------- total value len = 11.481792 GB avg = 0.209 KB (by entry num) total key len = 1.492593 GB avg = 0.027 KB total ukey len = 1.028941 GB avg = 0.015 KB total ukey num = 0.068139016 Billion total entry num = 0.054946745 Billion write speed all = 0.285647954 MB/sec (with seq num) write speed all = 0.275970162 MB/sec (without seq num) 2024/11/23-21:32:13.203872 139957643191872 [compaction_job.cc:1948] [default] [JOB 36224] Generated table #281679: 258859 keys, 83196900 bytes, temperature: kWarm 2024/11/23-21:32:13.203941 139957643191872 EVENT_LOG_v1 {"time_micros": 1732368733203905, "cf_name": "default", "job": 36224, "event": "table_file_creation", "file_number": 281679, "file_size": 83196900, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4273200905, "table_properties": {"data_size": 79975120, "index_size": 436096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 5177180, "raw_average_key_size": 20, "raw_value_size": 263258781, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 258859, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732368729, "oldest_key_time": 0, "file_creation_time": 1732368729, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 281679, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:32:13.263878 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281680.sst) /dev/shm/ToplingTemp/Topling-194742-ZTzcOE 2024/11/23-21:32:14.666574 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281680.sst): first pass time = 1.40's, 191.377'MB/sec 2024/11/23-21:32:16.154243 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281680.sst): old prealloc_size = 295279001, real_size = 80480896 2024/11/23-21:32:16.154420 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281680.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 258857 prefix = 4 raw-key = 0.0021 GB zip-key = 0.0004 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.2633 GB zip-val = 0.0800 GB avg-val =1017.01 avg-zval = 308.97 2024/11/23-21:32:16.154508 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281680.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0006 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:32:16.472836 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281680.sst): second pass time = 1.49's, 177.104'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.32's, 261.177'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2826.396'MB/sec (index lex order gen) rebuild zvType time = 0.23's, 0.280'MB/sec write SST data time = 0.09's, 950.759'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 2.690'MB, throughput = 86287.181'MB/sec zip my value time = 1.49's, unzip length = 0.263'GB zip my value throughput = 177.104'MB/sec zip pipeline throughput = 177.104'MB/sec entries = 258857 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.01 avg-zval = 308.97 usrkeys = 258857 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.40 avg-zval = 308.97 TagRS{ kinds = RS_Key0_TagN bytes = 40560 } TagArray{ size = 14464 bytes = 54256 } seq expand size = 101248 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0031 GB value = 0.2633 GB dict = 2.69 MB all = 0.2684 GB } __ZipSize{ index = 0.0004 GB value = 0.0800 GB dict = 2.69 MB all = 0.0832 GB } UnZip/Zip{ index = 7.1229 value = 3.2916 dict = 1.00 all = 3.2263 } Zip/UnZip{ index = 0.1404 value = 0.3038 dict = 1.00 all = 0.3100 } ---------------------------- total value len = 11.745152 GB avg = 0.213 KB (by entry num) total key len = 1.497770 GB avg = 0.027 KB total ukey len = 1.032047 GB avg = 0.015 KB total ukey num = 0.068397873 Billion total entry num = 0.055205602 Billion write speed all = 0.291538541 MB/sec (with seq num) write speed all = 0.281815876 MB/sec (without seq num) 2024/11/23-21:32:16.575202 139957643191872 [compaction_job.cc:1948] [default] [JOB 36224] Generated table #281680: 258857 keys, 83202960 bytes, temperature: kWarm 2024/11/23-21:32:16.575272 139957643191872 EVENT_LOG_v1 {"time_micros": 1732368736575235, "cf_name": "default", "job": 36224, "event": "table_file_creation", "file_number": 281680, "file_size": 83202960, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4273196189, "table_properties": {"data_size": 79979888, "index_size": 436096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 5177140, "raw_average_key_size": 20, "raw_value_size": 263259237, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 258857, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732368733, "oldest_key_time": 0, "file_creation_time": 1732368733, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 281680, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:32:16.639669 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281681.sst) /dev/shm/ToplingTemp/Topling-194742-xdsbLD 2024/11/23-21:32:18.516398 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281681.sst): first pass time = 1.88's, 143.037'MB/sec 2024/11/23-21:32:20.251363 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281681.sst): old prealloc_size = 295279001, real_size = 81077168 2024/11/23-21:32:20.251541 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281681.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 258858 prefix = 4 raw-key = 0.0021 GB zip-key = 0.0004 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.2633 GB zip-val = 0.0806 GB avg-val =1017.00 avg-zval = 311.28 2024/11/23-21:32:20.251683 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281681.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0006 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:32:20.580137 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281681.sst): second pass time = 1.73's, 151.848'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.33's, 255.806'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2746.975'MB/sec (index lex order gen) rebuild zvType time = 0.24's, 0.275'MB/sec write SST data time = 0.09's, 903.494'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 2.689'MB, throughput = 108906.727'MB/sec zip my value time = 1.73's, unzip length = 0.263'GB zip my value throughput = 151.848'MB/sec zip pipeline throughput = 151.848'MB/sec entries = 258858 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 311.28 usrkeys = 258858 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1019.55 avg-zval = 311.28 TagRS{ kinds = RS_Key0_TagN bytes = 40560 } TagArray{ size = 94139 bytes = 353040 } seq expand size = 658973 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0031 GB value = 0.2633 GB dict = 2.69 MB all = 0.2684 GB } __ZipSize{ index = 0.0004 GB value = 0.0806 GB dict = 2.69 MB all = 0.0841 GB } UnZip/Zip{ index = 7.1230 value = 3.2672 dict = 1.00 all = 3.1920 } Zip/UnZip{ index = 0.1404 value = 0.3061 dict = 1.00 all = 0.3133 } ---------------------------- total value len = 12.009070 GB avg = 0.217 KB (by entry num) total key len = 1.502947 GB avg = 0.027 KB total ukey len = 1.035154 GB avg = 0.015 KB total ukey num = 0.068656731 Billion total entry num = 0.055464460 Billion write speed all = 0.297435689 MB/sec (with seq num) write speed all = 0.287668319 MB/sec (without seq num) 2024/11/23-21:32:20.680504 139957643191872 [compaction_job.cc:1948] [default] [JOB 36224] Generated table #281681: 258858 keys, 84097336 bytes, temperature: kWarm 2024/11/23-21:32:20.680557 139957643191872 EVENT_LOG_v1 {"time_micros": 1732368740680525, "cf_name": "default", "job": 36224, "event": "table_file_creation", "file_number": 281681, "file_size": 84097336, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4273214595, "table_properties": {"data_size": 80576160, "index_size": 436096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 5177160, "raw_average_key_size": 20, "raw_value_size": 263259127, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 258858, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732368736, "oldest_key_time": 0, "file_creation_time": 1732368736, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 281681, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:32:20.741663 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281682.sst) /dev/shm/ToplingTemp/Topling-194742-XiT6SG 2024/11/23-21:32:22.384752 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281682.sst): first pass time = 1.64's, 163.376'MB/sec 2024/11/23-21:32:24.315875 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281682.sst): old prealloc_size = 295279001, real_size = 81104416 2024/11/23-21:32:24.316076 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281682.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 258863 prefix = 4 raw-key = 0.0021 GB zip-key = 0.0004 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.2633 GB zip-val = 0.0806 GB avg-val =1016.98 avg-zval = 311.37 2024/11/23-21:32:24.316232 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281682.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0006 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:32:24.640880 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281682.sst): second pass time = 1.93's, 136.431'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.32's, 258.860'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2393.226'MB/sec (index lex order gen) rebuild zvType time = 0.23's, 0.277'MB/sec write SST data time = 0.09's, 921.574'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 2.689'MB, throughput = 84621.207'MB/sec zip my value time = 1.93's, unzip length = 0.263'GB zip my value throughput = 136.431'MB/sec zip pipeline throughput = 165.568'MB/sec entries = 258863 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1016.98 avg-zval = 311.37 usrkeys = 258863 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1019.52 avg-zval = 311.37 TagRS{ kinds = RS_Key0_TagN bytes = 40560 } TagArray{ size = 93976 bytes = 352432 } seq expand size = 657832 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0031 GB value = 0.2633 GB dict = 2.69 MB all = 0.2684 GB } __ZipSize{ index = 0.0004 GB value = 0.0806 GB dict = 2.69 MB all = 0.0841 GB } UnZip/Zip{ index = 7.1231 value = 3.2661 dict = 1.00 all = 3.1910 } Zip/UnZip{ index = 0.1404 value = 0.3062 dict = 1.00 all = 0.3134 } ---------------------------- total value len = 12.272986 GB avg = 0.220 KB (by entry num) total key len = 1.508124 GB avg = 0.027 KB total ukey len = 1.038260 GB avg = 0.015 KB total ukey num = 0.068915594 Billion total entry num = 0.055723323 Billion write speed all = 0.303332043 MB/sec (with seq num) write speed all = 0.293519964 MB/sec (without seq num) 2024/11/23-21:32:24.744111 139957643191872 [compaction_job.cc:1948] [default] [JOB 36224] Generated table #281682: 258863 keys, 84123465 bytes, temperature: kWarm 2024/11/23-21:32:24.744170 139957643191872 EVENT_LOG_v1 {"time_micros": 1732368744744136, "cf_name": "default", "job": 36224, "event": "table_file_creation", "file_number": 281682, "file_size": 84123465, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4273215004, "table_properties": {"data_size": 80603408, "index_size": 436096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 5177260, "raw_average_key_size": 20, "raw_value_size": 263258384, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 258863, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732368740, "oldest_key_time": 0, "file_creation_time": 1732368740, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 281682, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:32:24.800930 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281683.sst) /dev/shm/ToplingTemp/Topling-194742-vNRI2G 2024/11/23-21:32:26.072321 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281683.sst): first pass time = 1.27's, 111.848'MB/sec 2024/11/23-21:32:27.141383 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281683.sst): old prealloc_size = 295279001, real_size = 42780752 2024/11/23-21:32:27.141552 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281683.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 137120 prefix = 4 raw-key = 0.0011 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1395 GB zip-val = 0.0425 GB avg-val =1017.04 avg-zval = 310.05 2024/11/23-21:32:27.141635 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281683.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.142 GB 2024/11/23-21:32:27.337510 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281683.sst): second pass time = 1.07's, 130.557'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.20's, 225.668'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1594.011'MB/sec (index lex order gen) rebuild zvType time = 0.14's, 0.237'MB/sec write SST data time = 0.05's, 861.665'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.459'MB, throughput = 50494.291'MB/sec zip my value time = 1.07's, unzip length = 0.139'GB zip my value throughput = 130.557'MB/sec zip pipeline throughput = 137.656'MB/sec entries = 137120 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.04 avg-zval = 310.05 usrkeys = 137120 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.42 avg-zval = 310.05 TagRS{ kinds = RS_Key0_TagN bytes = 21520 } TagArray{ size = 7453 bytes = 27968 } seq expand size = 52171 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1395 GB dict = 1.46 MB all = 0.1422 GB } __ZipSize{ index = 0.0002 GB value = 0.0425 GB dict = 1.46 MB all = 0.0443 GB } UnZip/Zip{ index = 7.1022 value = 3.2802 dict = 1.00 all = 3.2130 } Zip/UnZip{ index = 0.1408 value = 0.3049 dict = 1.00 all = 0.3112 } ---------------------------- total value len = 12.412495 GB avg = 0.222 KB (by entry num) total key len = 1.510867 GB avg = 0.027 KB total ukey len = 1.039905 GB avg = 0.015 KB total ukey num = 0.069052714 Billion total entry num = 0.055860443 Billion write speed all = 0.306444897 MB/sec (with seq num) write speed all = 0.296609256 MB/sec (without seq num) 2024/11/23-21:32:27.390155 139957643191872 [compaction_job.cc:1948] [default] [JOB 36224] Generated table #281683: 137120 keys, 44256980 bytes, temperature: kWarm 2024/11/23-21:32:27.390231 139957643191872 EVENT_LOG_v1 {"time_micros": 1732368747390188, "cf_name": "default", "job": 36224, "event": "table_file_creation", "file_number": 281683, "file_size": 44256980, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4273016836, "table_properties": {"data_size": 42514624, "index_size": 231680, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2742400, "raw_average_key_size": 20, "raw_value_size": 139456286, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 137120, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732368744, "oldest_key_time": 0, "file_creation_time": 1732368744, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 281683, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:32:27.438113 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281684.sst) /dev/shm/ToplingTemp/Topling-194742-ge1ysG 2024/11/23-21:32:31.378379 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281684.sst): first pass time = 3.94's, 68.127'MB/sec 2024/11/23-21:32:32.408287 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281684.sst): old prealloc_size = 295279001, real_size = 18631520 2024/11/23-21:32:32.412322 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281684.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZeroLengthBlobStore usrkeys = 9406654 prefix = 4 raw-key = 0.1556 GB zip-key = 0.0163 GB avg-key = 16.54 avg-zkey = 1.73 raw-val = 0.0000 GB zip-val = 0.0000 GB avg-val = 0.00 avg-zval = 0.00 2024/11/23-21:32:32.426017 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281684.sst: sumWaitingMem = 0.028 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0282 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:32:33.484049 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281684.sst): second pass time = 0.00's, 0.000'MB/sec, value only( 0.0% of KV) wait indexing time = 1.03's, remap KeyValue time = 1.08's, 49.018'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 4043.781'MB/sec (index lex order gen) rebuild zvType time = 0.03's, 74.822'MB/sec write SST data time = 1.04's, 50.689'MB/sec dict compress time = 0.00's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = -nan'MB/sec zip my value time = 0.00's, unzip length = 0.000'GB zip my value throughput = 0.000'MB/sec zip pipeline throughput = 0.000'MB/sec entries = 9406654 avg-key = 28.54 avg-zkey = 1.73 avg-val = 0.00 avg-zval = 0.00 usrkeys = 9406654 avg-key = 20.54 avg-zkey = 1.73 avg-val = 7.03 avg-zval = 0.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 9406654 bytes = 36450800 } seq expand size = 65898778 multi value expand size = 208800 cnt WriteAppend = 7 UnZipSize{ index = 0.1932 GB value = 0.0000 GB dict = 0.00 MB all = 0.2684 GB } __ZipSize{ index = 0.0163 GB value = 0.0000 GB dict = 0.00 MB all = 0.0527 GB } UnZip/Zip{ index = 11.8665 value = 0.0000 dict = 0.00 all = 5.0906 } Zip/UnZip{ index = 0.0843 value = inf dict = 0.00 all = 0.1964 } ---------------------------- total value len = 12.478602 GB avg = 0.191 KB (by entry num) total key len = 1.779302 GB avg = 0.027 KB total ukey len = 1.233088 GB avg = 0.016 KB total ukey num = 0.078459368 Billion total entry num = 0.065267097 Billion write speed all = 0.313765543 MB/sec (with seq num) write speed all = 0.302275178 MB/sec (without seq num) 2024/11/23-21:32:33.545623 139957643191872 [compaction_job.cc:1948] [default] [JOB 36224] Generated table #281684: 9406654 keys, 52731741 bytes, temperature: kWarm 2024/11/23-21:32:33.545691 139957643191872 EVENT_LOG_v1 {"time_micros": 1732368753545656, "cf_name": "default", "job": 36224, "event": "table_file_creation", "file_number": 281684, "file_size": 52731741, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237867995, "largest_seqno": 4273214897, "table_properties": {"data_size": 80, "index_size": 16279616, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 268435478, "raw_average_key_size": 28, "raw_value_size": 0, "raw_average_value_size": 0, "num_data_blocks": 1, "num_entries": 9406654, "num_filter_entries": 0, "num_deletions": 52200, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "fixed_value_len": 0, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZeroLengthBlobStore", "creation_time": 1732368747, "oldest_key_time": 0, "file_creation_time": 1732368747, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 281684, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:32:33.594728 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281685.sst) /dev/shm/ToplingTemp/Topling-194742-Zl6ZfG 2024/11/23-21:32:38.080042 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281685.sst): first pass time = 4.49's, 59.848'MB/sec 2024/11/23-21:32:39.063163 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281685.sst): old prealloc_size = 295279001, real_size = 18608864 2024/11/23-21:32:39.067075 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281685.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZeroLengthBlobStore usrkeys = 9393719 prefix = 4 raw-key = 0.1557 GB zip-key = 0.0163 GB avg-key = 16.58 avg-zkey = 1.73 raw-val = 0.0000 GB zip-val = 0.0000 GB avg-val = 0.00 avg-zval = 0.00 2024/11/23-21:32:39.079190 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281685.sst: sumWaitingMem = 0.028 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0282 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:32:40.116820 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281685.sst): second pass time = 0.00's, 0.000'MB/sec, value only( 0.0% of KV) wait indexing time = 0.98's, remap KeyValue time = 1.05's, 49.981'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 4194.065'MB/sec (index lex order gen) rebuild zvType time = 0.03's, 75.741'MB/sec write SST data time = 1.02's, 51.692'MB/sec dict compress time = 0.00's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = -nan'MB/sec zip my value time = 0.00's, unzip length = 0.000'GB zip my value throughput = 0.000'MB/sec zip pipeline throughput = 0.000'MB/sec entries = 9393719 avg-key = 28.58 avg-zkey = 1.73 avg-val = 0.00 avg-zval = 0.00 usrkeys = 9393719 avg-key = 20.58 avg-zkey = 1.73 avg-val = 7.01 avg-zval = 0.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 9393719 bytes = 36400672 } seq expand size = 65775833 multi value expand size = 79200 cnt WriteAppend = 7 UnZipSize{ index = 0.1933 GB value = 0.0000 GB dict = 0.00 MB all = 0.2684 GB } __ZipSize{ index = 0.0163 GB value = 0.0000 GB dict = 0.00 MB all = 0.0527 GB } UnZip/Zip{ index = 11.8871 value = 0.0000 dict = 0.00 all = 5.0973 } Zip/UnZip{ index = 0.0841 value = inf dict = 0.00 all = 0.1962 } ---------------------------- total value len = 12.544457 GB avg = 0.168 KB (by entry num) total key len = 2.047738 GB avg = 0.027 KB total ukey len = 1.426373 GB avg = 0.016 KB total ukey num = 0.087853087 Billion total entry num = 0.074660816 Billion write speed all = 0.321075218 MB/sec (with seq num) write speed all = 0.307932993 MB/sec (without seq num) 2024/11/23-21:32:40.177907 139957643191872 [compaction_job.cc:1948] [default] [JOB 36224] Generated table #281685: 9393719 keys, 52662173 bytes, temperature: kWarm 2024/11/23-21:32:40.177995 139957643191872 EVENT_LOG_v1 {"time_micros": 1732368760177942, "cf_name": "default", "job": 36224, "event": "table_file_creation", "file_number": 281685, "file_size": 52662173, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701061, "largest_seqno": 4012776349, "table_properties": {"data_size": 80, "index_size": 16260160, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 268435459, "raw_average_key_size": 28, "raw_value_size": 0, "raw_average_value_size": 0, "num_data_blocks": 1, "num_entries": 9393719, "num_filter_entries": 0, "num_deletions": 19800, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "fixed_value_len": 0, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZeroLengthBlobStore", "creation_time": 1732368753, "oldest_key_time": 0, "file_creation_time": 1732368753, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 281685, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:32:40.224837 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281686.sst) /dev/shm/ToplingTemp/Topling-194742-oIxf7G 2024/11/23-21:32:43.460285 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281686.sst): first pass time = 3.24's, 41.987'MB/sec 2024/11/23-21:32:43.943942 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281686.sst): old prealloc_size = 295279001, real_size = 9448928 2024/11/23-21:32:43.945776 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281686.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZeroLengthBlobStore usrkeys = 4772667 prefix = 4 raw-key = 0.0786 GB zip-key = 0.0083 GB avg-key = 16.46 avg-zkey = 1.73 raw-val = 0.0000 GB zip-val = 0.0000 GB avg-val = 0.00 avg-zval = 0.00 2024/11/23-21:32:43.951915 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281686.sst: sumWaitingMem = 0.014 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0137 GB, waited 0.000 sec, Key+Value = 0.136 GB 2024/11/23-21:32:44.488107 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281686.sst): second pass time = 0.00's, 0.000'MB/sec, value only( 0.0% of KV) wait indexing time = 0.48's, remap KeyValue time = 0.54's, 49.160'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 4524.063'MB/sec (index lex order gen) rebuild zvType time = 0.02's, 73.111'MB/sec write SST data time = 0.53's, 50.856'MB/sec dict compress time = 0.00's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = -nan'MB/sec zip my value time = 0.00's, unzip length = 0.000'GB zip my value throughput = 0.000'MB/sec zip pipeline throughput = 0.000'MB/sec entries = 4772667 avg-key = 28.46 avg-zkey = 1.73 avg-val = 0.00 avg-zval = 0.00 usrkeys = 4772667 avg-key = 20.46 avg-zkey = 1.73 avg-val = 7.03 avg-zval = 0.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 4772667 bytes = 18494096 } seq expand size = 33435669 multi value expand size = 108000 cnt WriteAppend = 7 UnZipSize{ index = 0.0977 GB value = 0.0000 GB dict = 0.00 MB all = 0.1358 GB } __ZipSize{ index = 0.0083 GB value = 0.0000 GB dict = 0.00 MB all = 0.0268 GB } UnZip/Zip{ index = 11.8299 value = 0.0000 dict = 0.00 all = 5.0781 } Zip/UnZip{ index = 0.0845 value = inf dict = 0.00 all = 0.1969 } ---------------------------- total value len = 12.578001 GB avg = 0.158 KB (by entry num) total key len = 2.183581 GB avg = 0.027 KB total ukey len = 1.524035 GB avg = 0.016 KB total ukey num = 0.092625754 Billion total entry num = 0.079433483 Billion write speed all = 0.324771038 MB/sec (with seq num) write speed all = 0.310790047 MB/sec (without seq num) 2024/11/23-21:32:44.522477 139957643191872 [compaction_job.cc:1948] [default] [JOB 36224] Generated table #281686: 4772667 keys, 26750940 bytes, temperature: kWarm 2024/11/23-21:32:44.522545 139957643191872 EVENT_LOG_v1 {"time_micros": 1732368764522510, "cf_name": "default", "job": 36224, "event": "table_file_creation", "file_number": 281686, "file_size": 26750940, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237869311, "largest_seqno": 4273214991, "table_properties": {"data_size": 80, "index_size": 8255488, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 135843303, "raw_average_key_size": 28, "raw_value_size": 0, "raw_average_value_size": 0, "num_data_blocks": 1, "num_entries": 4772667, "num_filter_entries": 0, "num_deletions": 27000, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "fixed_value_len": 0, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZeroLengthBlobStore", "creation_time": 1732368760, "oldest_key_time": 0, "file_creation_time": 1732368760, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 281686, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:32:44.558389 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281687.sst) /dev/shm/ToplingTemp/Topling-194742-o74y2E 2024/11/23-21:32:48.717325 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281687.sst): first pass time = 4.16's, 64.545'MB/sec 2024/11/23-21:32:49.963265 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281687.sst): old prealloc_size = 295279001, real_size = 30874400 2024/11/23-21:32:49.971001 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281687.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZeroLengthBlobStore usrkeys = 9256396 prefix = 4 raw-key = 0.1574 GB zip-key = 0.0286 GB avg-key = 17.00 avg-zkey = 3.09 raw-val = 0.0000 GB zip-val = 0.0000 GB avg-val = 0.00 avg-zval = 0.00 2024/11/23-21:32:49.984046 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281687.sst: sumWaitingMem = 0.028 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0278 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:32:51.337928 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281687.sst): second pass time = 0.00's, 0.000'MB/sec, value only( 0.0% of KV) wait indexing time = 1.25's, remap KeyValue time = 1.37's, 46.116'MB/sec (all stages of remap) Get OrderMap time = 0.01's, 3702.912'MB/sec (index lex order gen) rebuild zvType time = 0.04's, 65.522'MB/sec write SST data time = 1.33's, 47.606'MB/sec dict compress time = 0.00's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = -nan'MB/sec zip my value time = 0.00's, unzip length = 0.000'GB zip my value throughput = 0.000'MB/sec zip pipeline throughput = 0.000'MB/sec entries = 9256396 avg-key = 29.00 avg-zkey = 3.09 avg-val = 0.00 avg-zval = 0.00 usrkeys = 9256396 avg-key = 21.00 avg-zkey = 3.09 avg-val = 7.00 avg-zval = 0.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 9256396 bytes = 34711504 } seq expand size = 64794772 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.1944 GB value = 0.0000 GB dict = 0.00 MB all = 0.2684 GB } __ZipSize{ index = 0.0286 GB value = 0.0000 GB dict = 0.00 MB all = 0.0634 GB } UnZip/Zip{ index = 6.8062 value = 0.0000 dict = 0.00 all = 4.2344 } Zip/UnZip{ index = 0.1469 value = inf dict = 0.00 all = 0.2362 } ---------------------------- total value len = 12.642796 GB avg = 0.143 KB (by entry num) total key len = 2.452016 GB avg = 0.028 KB total ukey len = 1.718420 GB avg = 0.017 KB total ukey num = 0.101882150 Billion total entry num = 0.088689879 Billion write speed all = 0.332052429 MB/sec (with seq num) write speed all = 0.316444582 MB/sec (without seq num) 2024/11/23-21:32:51.411062 139957643191872 [compaction_job.cc:1948] [default] [JOB 36224] Generated table #281687: 9256396 keys, 63393536 bytes, temperature: kWarm 2024/11/23-21:32:51.411133 139957643191872 EVENT_LOG_v1 {"time_micros": 1732368771411096, "cf_name": "default", "job": 36224, "event": "table_file_creation", "file_number": 281687, "file_size": 63393536, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237867844, "largest_seqno": 4273214898, "table_properties": {"data_size": 80, "index_size": 28560064, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 268435484, "raw_average_key_size": 29, "raw_value_size": 0, "raw_average_value_size": 0, "num_data_blocks": 1, "num_entries": 9256396, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 29, "fixed_value_len": 0, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZeroLengthBlobStore", "creation_time": 1732368764, "oldest_key_time": 0, "file_creation_time": 1732368764, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 281687, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:32:51.458932 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281688.sst) /dev/shm/ToplingTemp/Topling-194742-R5hjoF 2024/11/23-21:32:56.416650 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281688.sst): first pass time = 4.96's, 54.145'MB/sec 2024/11/23-21:32:57.625697 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281688.sst): old prealloc_size = 295279001, real_size = 30552224 2024/11/23-21:32:57.633781 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281688.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZeroLengthBlobStore usrkeys = 9256396 prefix = 4 raw-key = 0.1574 GB zip-key = 0.0282 GB avg-key = 17.00 avg-zkey = 3.05 raw-val = 0.0000 GB zip-val = 0.0000 GB avg-val = 0.00 avg-zval = 0.00 2024/11/23-21:32:57.646885 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281688.sst: sumWaitingMem = 0.028 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0278 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:32:58.992547 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281688.sst): second pass time = 0.00's, 0.000'MB/sec, value only( 0.0% of KV) wait indexing time = 1.21's, remap KeyValue time = 1.37's, 46.144'MB/sec (all stages of remap) Get OrderMap time = 0.01's, 3500.557'MB/sec (index lex order gen) rebuild zvType time = 0.03's, 66.409'MB/sec write SST data time = 1.32's, 47.639'MB/sec dict compress time = 0.00's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = -nan'MB/sec zip my value time = 0.00's, unzip length = 0.000'GB zip my value throughput = 0.000'MB/sec zip pipeline throughput = 0.000'MB/sec entries = 9256396 avg-key = 29.00 avg-zkey = 3.05 avg-val = 0.00 avg-zval = 0.00 usrkeys = 9256396 avg-key = 21.00 avg-zkey = 3.05 avg-val = 7.00 avg-zval = 0.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 9256396 bytes = 34711504 } seq expand size = 64794772 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.1944 GB value = 0.0000 GB dict = 0.00 MB all = 0.2684 GB } __ZipSize{ index = 0.0282 GB value = 0.0000 GB dict = 0.00 MB all = 0.0631 GB } UnZip/Zip{ index = 6.8838 value = 0.0000 dict = 0.00 all = 4.2561 } Zip/UnZip{ index = 0.1453 value = inf dict = 0.00 all = 0.2350 } ---------------------------- total value len = 12.707591 GB avg = 0.130 KB (by entry num) total key len = 2.720452 GB avg = 0.028 KB total ukey len = 1.912804 GB avg = 0.017 KB total ukey num = 0.111138546 Billion total entry num = 0.097946275 Billion write speed all = 0.339325619 MB/sec (with seq num) write speed all = 0.322091713 MB/sec (without seq num) 2024/11/23-21:32:59.065406 139957643191872 [compaction_job.cc:1948] [default] [JOB 36224] Generated table #281688: 9256396 keys, 63071360 bytes, temperature: kWarm 2024/11/23-21:32:59.065473 139957643191872 EVENT_LOG_v1 {"time_micros": 1732368779065437, "cf_name": "default", "job": 36224, "event": "table_file_creation", "file_number": 281688, "file_size": 63071360, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273700904, "largest_seqno": 4012775807, "table_properties": {"data_size": 80, "index_size": 28237888, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 268435484, "raw_average_key_size": 29, "raw_value_size": 0, "raw_average_value_size": 0, "num_data_blocks": 1, "num_entries": 9256396, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 29, "fixed_value_len": 0, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZeroLengthBlobStore", "creation_time": 1732368771, "oldest_key_time": 0, "file_creation_time": 1732368771, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 281688, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:32:59.115606 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281689.sst) /dev/shm/ToplingTemp/Topling-194742-IZnidH 2024/11/23-21:33:01.347099 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281689.sst): first pass time = 2.23's, 63.190'MB/sec 2024/11/23-21:33:02.011241 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281689.sst): old prealloc_size = 295279001, real_size = 16244320 2024/11/23-21:33:02.015516 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281689.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZeroLengthBlobStore usrkeys = 4862248 prefix = 4 raw-key = 0.0827 GB zip-key = 0.0150 GB avg-key = 17.00 avg-zkey = 3.09 raw-val = 0.0000 GB zip-val = 0.0000 GB avg-val = 0.00 avg-zval = 0.00 2024/11/23-21:33:02.022338 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281689.sst: sumWaitingMem = 0.014 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0140 GB, waited 0.000 sec, Key+Value = 0.141 GB 2024/11/23-21:33:02.699894 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00/281689.sst): second pass time = 0.00's, 0.000'MB/sec, value only( 0.0% of KV) wait indexing time = 0.66's, remap KeyValue time = 0.69's, 48.400'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3522.598'MB/sec (index lex order gen) rebuild zvType time = 0.02's, 66.802'MB/sec write SST data time = 0.67's, 50.032'MB/sec dict compress time = 0.00's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = -nan'MB/sec zip my value time = 0.00's, unzip length = 0.000'GB zip my value throughput = 0.000'MB/sec zip pipeline throughput = 0.000'MB/sec entries = 4862248 avg-key = 29.00 avg-zkey = 3.09 avg-val = 0.00 avg-zval = 0.00 usrkeys = 4862248 avg-key = 21.00 avg-zkey = 3.09 avg-val = 7.00 avg-zval = 0.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 4862248 bytes = 18233440 } seq expand size = 34035736 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.1021 GB value = 0.0000 GB dict = 0.00 MB all = 0.1410 GB } __ZipSize{ index = 0.0150 GB value = 0.0000 GB dict = 0.00 MB all = 0.0333 GB } UnZip/Zip{ index = 6.7942 value = 0.0000 dict = 0.00 all = 4.2304 } Zip/UnZip{ index = 0.1472 value = inf dict = 0.00 all = 0.2364 } ---------------------------- total value len = 12.741626 GB avg = 0.124 KB (by entry num) total key len = 2.861457 GB avg = 0.028 KB total ukey len = 2.014911 GB avg = 0.017 KB total ukey num = 0.116000794 Billion total entry num = 0.102808523 Billion write speed all = 0.343147503 MB/sec (with seq num) write speed all = 0.325059546 MB/sec (without seq num) 2024/11/23-21:33:02.741696 139957643191872 [compaction_job.cc:1948] [default] [JOB 36224] Generated table #281689: 4862248 keys, 33331519 bytes, temperature: kWarm 2024/11/23-21:33:02.741767 139957643191872 EVENT_LOG_v1 {"time_micros": 1732368782741730, "cf_name": "default", "job": 36224, "event": "table_file_creation", "file_number": 281689, "file_size": 33331519, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237867819, "largest_seqno": 4273214992, "table_properties": {"data_size": 80, "index_size": 15028480, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 141005192, "raw_average_key_size": 29, "raw_value_size": 0, "raw_average_value_size": 0, "num_data_blocks": 1, "num_entries": 4862248, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 29, "fixed_value_len": 0, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZeroLengthBlobStore", "creation_time": 1732368779, "oldest_key_time": 0, "file_creation_time": 1732368779, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 281689, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:33:02.767302 139957643191872 [compaction_job.cc:703] job-36224: subcompact[0], size: 1.419828 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36224/att-00 [281670,281671,281672,281673,281674,281675,281676,281677,281678,281679,281680,281681,281682,281683,281684,281685,281686,281687,281688,281689] 2024/11/23-21:33:02.980162 139957643191872 (Original Log Time 2024/11/23-21:31:39.629332) EVENT_LOG_v1 {"time_micros": 1732368699629296, "job": 36224, "event": "compaction_started", "compaction_reason": "ManualCompaction", "files_L3": [281664, 281665, 281666, 281667, 281599, 281600, 281588, 281593, 281595, 281601, 281602], "files_L4": [281657, 281658, 281659, 281660, 281661, 209784, 209785, 209786, 209787, 210856, 210858, 210857, 210863, 211883, 211884, 211885, 211886, 276543, 276544, 212835, 212832, 261641, 261642, 260306, 280232, 270877, 270878, 265852], "score": -1, "input_data_size": 1516678761, "oldest_snapshot_seqno": 4274435414} 2024/11/23-21:33:02.980165 139957643191872 (Original Log Time 2024/11/23-21:33:02.978575) [compaction_job.cc:2022] [default] [JOB 36224] Compacted 11@3 + 28@4 files to L4 => 1419827668 bytes 2024/11/23-21:33:02.980166 139957643191872 (Original Log Time 2024/11/23-21:33:02.980083) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 0 20 0 0] max score 0.01, MB/sec: 18.2 rd, 17.1 wr, level 4, files in(11, 28) out(20 +0 blob) MB in(236.5, 1209.9 +0.0 blob) out(1354.1 +0.0 blob), read-write-amplify(11.8) write-amplify(5.7) OK, records in: 51725604, records dropped: 1275245 output_compression: Snappy 2024/11/23-21:33:02.980170 139957643191872 (Original Log Time 2024/11/23-21:33:02.980135) EVENT_LOG_v1 {"time_micros": 1732368782980113, "cf": "default", "job": 36224, "event": "compaction_finished", "compaction_time_micros": 83137984, "compaction_time_cpu_micros": 36376654, "output_level": 4, "num_output_files": 20, "total_output_size": 1419827668, "num_input_records": 51725604, "num_output_records": 50450359, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 938380693, "file_range_sync_nanos": 0, "file_fsync_nanos": 13990, "file_prepare_write_nanos": 45010, "lsm_state": [0, 0, 0, 0, 20, 0, 0]} 2024/11/23-21:33:03.108302 139957643191872 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732368699,"executesMs":83488,"compactionJobId":36224,"attempt":0,"compactionInputRawBytes":5294889624,"compactionInputZipBytes":1516678761,"compactionOutputRawBytes":0,"compactionOutputZipBytes":1419827668,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-b5a2b1435ff38526ca12930fe435c505-493a4ad9bfc9471f-00"}}, response = 2024/11/23-21:33:03.108381 139957643191872 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-36224/att-00: olev 4, work 83.489 s, result 7.743 ms, install 1.610 ms, input{raw 4.931 GiB zip 1.413 GiB}