2024/11/23-21:54:08.608685 139957273912896 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00: fork to child time = 0.007624 sec 2024/11/23-21:54:08.609465 139957273912896 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00 2024/11/23-21:54:08.617431 139957273912896 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00 2024/11/23-21:54:08.618365 139957273912896 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-21:54:08.618682 139957273912896 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-21:54:08.671083 139957273912896 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-36558/att-00/MANIFEST-206347 2024/11/23-21:54:08.702490 139957273912896 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-36558/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 282237, last_sequence is 4284648848, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 282076 2024/11/23-21:54:08.702497 139957273912896 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-21:54:08.702697 139957273912896 [version_set.cc:5721] Creating manifest 282237 2024/11/23-21:54:08.739770 139957273912896 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00: bottommost_level: fake = 1, rpc = 1 2024/11/23-21:54:08.777729 139957273912896 [compaction_job.cc:2437] [default] [JOB 36558] Compacting 60@4 files to L5, score -1.00, subcompactions 1 : 1 2024/11/23-21:54:08.777758 139957273912896 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 27371 Base level 4, inputs: [281026(70MB) 280966(70MB) 281027(70MB) 281025(58MB) 281028(59MB) 281029(59MB) 281030(58MB) 281031(58MB) 281092(58MB) 281093(59MB) 281094(59MB) 281095(63MB) 281096(52MB) 281097(54MB) 281098(58MB) 281099(57MB) 281100(54MB) 281101(58MB) 281102(58MB) 281103(54MB) 281104(57MB) 281105(59MB) 281106(53MB) 281107(55MB) 281108(59MB) 281109(55MB) 281110(17MB) 281111(6613KB) 281361(79MB) 281362(79MB) 281363(79MB) 281364(79MB) 281365(79MB) 281366(79MB) 281367(79MB) 281368(79MB) 281369(79MB) 281370(79MB) 281371(79MB) 281372(79MB) 281373(79MB) 281374(79MB) 281375(79MB) 281376(79MB) 281377(79MB) 281378(79MB) 281379(79MB) 281380(79MB) 281381(79MB) 281382(79MB) 281383(79MB) 281384(79MB) 281385(79MB) 281386(79MB) 281387(79MB) 281388(79MB) 281389(79MB) 281390(79MB) 281391(79MB) 281392(79MB)] 2024/11/23-21:54:08.783517 139957273912896 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-21:54:08.783546 139957273912896 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-21:54:08.783567 139957273912896 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-21:54:08.783578 139957273912896 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-21:54:08.783583 139957273912896 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-21:54:09.681660 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282238.sst) /dev/shm/ToplingTemp/Topling-199246-xJWu4g 2024/11/23-21:54:13.579722 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282238.sst): first pass time = 3.90's, 137.729'MB/sec 2024/11/23-21:54:19.762764 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282238.sst): old prealloc_size = 590558003, real_size = 147351984 2024/11/23-21:54:19.764932 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282238.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 3706898 prefix = 4 raw-key = 0.0593 GB zip-key = 0.0070 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.4331 GB zip-val = 0.1394 GB avg-val = 116.83 avg-zval = 37.61 2024/11/23-21:54:19.765004 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282238.sst: sumWaitingMem = 0.010 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0102 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:54:22.662788 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282238.sst): second pass time = 6.18's, 70.070'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.90's, 51.993'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3282.421'MB/sec (index lex order gen) rebuild zvType time = 2.40's, 0.386'MB/sec write SST data time = 0.50's, 302.253'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.321'MB, throughput = 123610.231'MB/sec zip my value time = 6.18's, unzip length = 0.433'GB zip my value throughput = 70.070'MB/sec zip pipeline throughput = 70.070'MB/sec entries = 3706898 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.83 avg-zval = 37.61 usrkeys = 3706898 avg-key = 20.00 avg-zkey = 1.89 avg-val = 116.83 avg-zval = 37.61 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0741 GB value = 0.4331 GB dict = 4.32 MB all = 0.5369 GB } __ZipSize{ index = 0.0070 GB value = 0.1394 GB dict = 4.32 MB all = 0.1508 GB } UnZip/Zip{ index = 10.5890 value = 3.1062 dict = 1.00 all = 3.5606 } Zip/UnZip{ index = 0.0944 value = 0.3219 dict = 1.00 all = 0.2809 } ---------------------------- total value len = 9.281698 GB avg = 0.166 KB (by entry num) total key len = 1.544614 GB avg = 0.028 KB total ukey len = 1.072016 GB avg = 0.015 KB total ukey num = 0.069257333 Billion total entry num = 0.056065062 Billion write speed all = 0.231576644 MB/sec (with seq num) write speed all = 0.221982715 MB/sec (without seq num) 2024/11/23-21:54:22.827420 139957273912896 [compaction_job.cc:1948] [default] [JOB 36558] Generated table #282238: 3706898 keys, 150780454 bytes, temperature: kWarm 2024/11/23-21:54:22.827526 139957273912896 EVENT_LOG_v1 {"time_micros": 1732370062827462, "cf_name": "default", "job": 36558, "event": "table_file_creation", "file_number": 282238, "file_size": 150780454, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 139423648, "index_size": 7001408, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 103793144, "raw_average_key_size": 28, "raw_value_size": 433077888, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 3706898, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370049, "oldest_key_time": 0, "file_creation_time": 1732370049, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282238, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:54:22.922559 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282239.sst) /dev/shm/ToplingTemp/Topling-199246-Os7vGd 2024/11/23-21:54:25.606823 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282239.sst): first pass time = 2.68's, 200.010'MB/sec 2024/11/23-21:54:30.998580 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282239.sst): old prealloc_size = 590558003, real_size = 132992752 2024/11/23-21:54:31.000584 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282239.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 3695561 prefix = 4 raw-key = 0.0591 GB zip-key = 0.0070 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.4334 GB zip-val = 0.1251 GB avg-val = 117.27 avg-zval = 33.85 2024/11/23-21:54:31.000632 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282239.sst: sumWaitingMem = 0.010 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0102 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:54:33.962485 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282239.sst): second pass time = 5.39's, 80.409'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.96's, 46.030'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3553.128'MB/sec (index lex order gen) rebuild zvType time = 2.45's, 0.378'MB/sec write SST data time = 0.51's, 264.913'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.326'MB, throughput = 125871.788'MB/sec zip my value time = 5.39's, unzip length = 0.433'GB zip my value throughput = 80.409'MB/sec zip pipeline throughput = 80.409'MB/sec entries = 3695561 avg-key = 28.00 avg-zkey = 1.89 avg-val = 117.27 avg-zval = 33.85 usrkeys = 3695561 avg-key = 20.00 avg-zkey = 1.89 avg-val = 117.27 avg-zval = 33.85 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0739 GB value = 0.4334 GB dict = 4.33 MB all = 0.5369 GB } __ZipSize{ index = 0.0070 GB value = 0.1251 GB dict = 4.33 MB all = 0.1364 GB } UnZip/Zip{ index = 10.5890 value = 3.4647 dict = 1.00 all = 3.9352 } Zip/UnZip{ index = 0.0944 value = 0.2886 dict = 1.00 all = 0.2541 } ---------------------------- total value len = 9.715093 GB avg = 0.163 KB (by entry num) total key len = 1.648090 GB avg = 0.028 KB total ukey len = 1.145927 GB avg = 0.016 KB total ukey num = 0.072952894 Billion total entry num = 0.059760623 Billion write speed all = 0.243001672 MB/sec (with seq num) write speed all = 0.232777824 MB/sec (without seq num) 2024/11/23-21:54:34.111823 139957273912896 [compaction_job.cc:1948] [default] [JOB 36558] Generated table #282239: 3695561 keys, 136428950 bytes, temperature: kWarm 2024/11/23-21:54:34.111898 139957273912896 EVENT_LOG_v1 {"time_micros": 1732370074111861, "cf_name": "default", "job": 36558, "event": "table_file_creation", "file_number": 282239, "file_size": 136428950, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 125088672, "index_size": 6980032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 103475708, "raw_average_key_size": 28, "raw_value_size": 433395290, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 3695561, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370062, "oldest_key_time": 0, "file_creation_time": 1732370062, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282239, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:54:34.198029 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282240.sst) /dev/shm/ToplingTemp/Topling-199246-GgZBke 2024/11/23-21:54:37.698711 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282240.sst): first pass time = 3.50's, 153.364'MB/sec 2024/11/23-21:54:42.061866 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282240.sst): old prealloc_size = 590558003, real_size = 117062176 2024/11/23-21:54:42.063928 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282240.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 3686047 prefix = 4 raw-key = 0.0590 GB zip-key = 0.0070 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.4337 GB zip-val = 0.1091 GB avg-val = 117.65 avg-zval = 29.61 2024/11/23-21:54:42.063993 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282240.sst: sumWaitingMem = 0.010 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0101 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:54:45.071272 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282240.sst): second pass time = 4.36's, 99.437'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 3.01's, 40.037'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3462.901'MB/sec (index lex order gen) rebuild zvType time = 2.49's, 0.370'MB/sec write SST data time = 0.52's, 233.293'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.327'MB, throughput = 124574.118'MB/sec zip my value time = 4.36's, unzip length = 0.434'GB zip my value throughput = 99.437'MB/sec zip pipeline throughput = 99.437'MB/sec entries = 3686047 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.65 avg-zval = 29.61 usrkeys = 3686047 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.65 avg-zval = 29.61 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0737 GB value = 0.4337 GB dict = 4.33 MB all = 0.5369 GB } __ZipSize{ index = 0.0070 GB value = 0.1091 GB dict = 4.33 MB all = 0.1205 GB } UnZip/Zip{ index = 10.5196 value = 3.9737 dict = 1.00 all = 4.4558 } Zip/UnZip{ index = 0.0951 value = 0.2517 dict = 1.00 all = 0.2244 } ---------------------------- total value len = 10.148755 GB avg = 0.160 KB (by entry num) total key len = 1.751299 GB avg = 0.028 KB total ukey len = 1.219648 GB avg = 0.016 KB total ukey num = 0.076638941 Billion total entry num = 0.063446670 Billion write speed all = 0.254422218 MB/sec (with seq num) write speed all = 0.243570340 MB/sec (without seq num) 2024/11/23-21:54:45.214567 139957273912896 [compaction_job.cc:1948] [default] [JOB 36558] Generated table #282240: 3686047 keys, 120487344 bytes, temperature: kWarm 2024/11/23-21:54:45.214638 139957273912896 EVENT_LOG_v1 {"time_micros": 1732370085214603, "cf_name": "default", "job": 36558, "event": "table_file_creation", "file_number": 282240, "file_size": 120487344, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 109132560, "index_size": 7007936, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 103209316, "raw_average_key_size": 28, "raw_value_size": 433661726, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 3686047, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370074, "oldest_key_time": 0, "file_creation_time": 1732370074, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282240, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:54:45.309648 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282241.sst) /dev/shm/ToplingTemp/Topling-199246-ZYru8g 2024/11/23-21:54:48.669478 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282241.sst): first pass time = 3.36's, 159.793'MB/sec 2024/11/23-21:54:53.103785 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282241.sst): old prealloc_size = 590558003, real_size = 117209744 2024/11/23-21:54:53.105963 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282241.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 3690781 prefix = 4 raw-key = 0.0591 GB zip-key = 0.0071 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.4335 GB zip-val = 0.1092 GB avg-val = 117.46 avg-zval = 29.59 2024/11/23-21:54:53.106020 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282241.sst: sumWaitingMem = 0.010 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0101 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:54:56.088191 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282241.sst): second pass time = 4.43's, 97.812'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.98's, 40.421'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3290.604'MB/sec (index lex order gen) rebuild zvType time = 2.47's, 0.374'MB/sec write SST data time = 0.51's, 234.954'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.326'MB, throughput = 130357.291'MB/sec zip my value time = 4.43's, unzip length = 0.434'GB zip my value throughput = 97.812'MB/sec zip pipeline throughput = 97.812'MB/sec entries = 3690781 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.46 avg-zval = 29.59 usrkeys = 3690781 avg-key = 20.00 avg-zkey = 1.91 avg-val = 117.46 avg-zval = 29.59 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0738 GB value = 0.4335 GB dict = 4.33 MB all = 0.5369 GB } __ZipSize{ index = 0.0071 GB value = 0.1092 GB dict = 4.33 MB all = 0.1206 GB } UnZip/Zip{ index = 10.4513 value = 3.9692 dict = 1.00 all = 4.4505 } Zip/UnZip{ index = 0.0957 value = 0.2519 dict = 1.00 all = 0.2247 } ---------------------------- total value len = 10.582284 GB avg = 0.158 KB (by entry num) total key len = 1.854641 GB avg = 0.028 KB total ukey len = 1.293464 GB avg = 0.016 KB total ukey num = 0.080329722 Billion total entry num = 0.067137451 Billion write speed all = 0.265837861 MB/sec (with seq num) write speed all = 0.254357418 MB/sec (without seq num) 2024/11/23-21:54:56.226781 139957273912896 [compaction_job.cc:1948] [default] [JOB 36558] Generated table #282241: 3690781 keys, 120632276 bytes, temperature: kWarm 2024/11/23-21:54:56.226847 139957273912896 EVENT_LOG_v1 {"time_micros": 1732370096226812, "cf_name": "default", "job": 36558, "event": "table_file_creation", "file_number": 282241, "file_size": 120632276, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 109224064, "index_size": 7062784, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 103341868, "raw_average_key_size": 28, "raw_value_size": 433529054, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 3690781, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370085, "oldest_key_time": 0, "file_creation_time": 1732370085, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282241, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:54:56.312655 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282242.sst) /dev/shm/ToplingTemp/Topling-199246-inBjnd 2024/11/23-21:54:59.836441 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282242.sst): first pass time = 3.52's, 152.358'MB/sec 2024/11/23-21:55:04.275914 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282242.sst): old prealloc_size = 590558003, real_size = 116419488 2024/11/23-21:55:04.278040 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282242.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 3686644 prefix = 4 raw-key = 0.0590 GB zip-key = 0.0070 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.4336 GB zip-val = 0.1085 GB avg-val = 117.63 avg-zval = 29.43 2024/11/23-21:55:04.278090 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282242.sst: sumWaitingMem = 0.010 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0101 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:55:07.261583 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282242.sst): second pass time = 4.44's, 97.728'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.99's, 40.140'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3364.007'MB/sec (index lex order gen) rebuild zvType time = 2.48's, 0.371'MB/sec write SST data time = 0.50's, 238.505'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.327'MB, throughput = 92641.244'MB/sec zip my value time = 4.44's, unzip length = 0.434'GB zip my value throughput = 97.728'MB/sec zip pipeline throughput = 97.728'MB/sec entries = 3686644 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.63 avg-zval = 29.43 usrkeys = 3686644 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.63 avg-zval = 29.43 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0737 GB value = 0.4336 GB dict = 4.33 MB all = 0.5369 GB } __ZipSize{ index = 0.0070 GB value = 0.1085 GB dict = 4.33 MB all = 0.1198 GB } UnZip/Zip{ index = 10.5200 value = 3.9971 dict = 1.00 all = 4.4797 } Zip/UnZip{ index = 0.0951 value = 0.2502 dict = 1.00 all = 0.2232 } ---------------------------- total value len = 11.015929 GB avg = 0.156 KB (by entry num) total key len = 1.957867 GB avg = 0.028 KB total ukey len = 1.367197 GB avg = 0.016 KB total ukey num = 0.084016366 Billion total entry num = 0.070824095 Billion write speed all = 0.277247203 MB/sec (with seq num) write speed all = 0.265139238 MB/sec (without seq num) 2024/11/23-21:55:07.396270 139957273912896 [compaction_job.cc:1948] [default] [JOB 36558] Generated table #282242: 3686644 keys, 119844006 bytes, temperature: kWarm 2024/11/23-21:55:07.396339 139957273912896 EVENT_LOG_v1 {"time_micros": 1732370107396305, "cf_name": "default", "job": 36558, "event": "table_file_creation", "file_number": 282242, "file_size": 119844006, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 108488784, "index_size": 7008832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 103226032, "raw_average_key_size": 28, "raw_value_size": 433644968, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 3686644, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370096, "oldest_key_time": 0, "file_creation_time": 1732370096, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282242, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:55:07.489402 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282243.sst) /dev/shm/ToplingTemp/Topling-199246-AUj4nh 2024/11/23-21:55:10.814078 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282243.sst): first pass time = 3.32's, 161.483'MB/sec 2024/11/23-21:55:15.325454 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282243.sst): old prealloc_size = 590558003, real_size = 121835360 2024/11/23-21:55:15.327603 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282243.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 3688840 prefix = 4 raw-key = 0.0590 GB zip-key = 0.0070 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.4336 GB zip-val = 0.1139 GB avg-val = 117.54 avg-zval = 30.88 2024/11/23-21:55:15.327649 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282243.sst: sumWaitingMem = 0.010 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0101 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:55:18.383618 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282243.sst): second pass time = 4.51's, 96.152'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 3.06's, 40.964'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3326.128'MB/sec (index lex order gen) rebuild zvType time = 2.56's, 0.361'MB/sec write SST data time = 0.50's, 250.498'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.327'MB, throughput = 135059.121'MB/sec zip my value time = 4.51's, unzip length = 0.434'GB zip my value throughput = 96.152'MB/sec zip pipeline throughput = 96.152'MB/sec entries = 3688840 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.54 avg-zval = 30.88 usrkeys = 3688840 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.54 avg-zval = 30.88 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 10 UnZipSize{ index = 0.0738 GB value = 0.4336 GB dict = 4.33 MB all = 0.5369 GB } __ZipSize{ index = 0.0070 GB value = 0.1139 GB dict = 4.33 MB all = 0.1253 GB } UnZip/Zip{ index = 10.5192 value = 3.8067 dict = 1.00 all = 4.2856 } Zip/UnZip{ index = 0.0951 value = 0.2627 dict = 1.00 all = 0.2333 } ---------------------------- total value len = 11.449512 GB avg = 0.154 KB (by entry num) total key len = 2.061155 GB avg = 0.028 KB total ukey len = 1.440973 GB avg = 0.016 KB total ukey num = 0.087705206 Billion total entry num = 0.074512935 Billion write speed all = 0.288651412 MB/sec (with seq num) write speed all = 0.275915837 MB/sec (without seq num) 2024/11/23-21:55:18.522244 139957273912896 [compaction_job.cc:1948] [default] [JOB 36558] Generated table #282243: 3688840 keys, 125274216 bytes, temperature: kWarm 2024/11/23-21:55:18.522313 139957273912896 EVENT_LOG_v1 {"time_micros": 1732370118522277, "cf_name": "default", "job": 36558, "event": "table_file_creation", "file_number": 282243, "file_size": 125274216, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 113899472, "index_size": 7013504, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 103287520, "raw_average_key_size": 28, "raw_value_size": 433583392, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 3688840, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370107, "oldest_key_time": 0, "file_creation_time": 1732370107, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282243, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:55:18.621279 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282244.sst) /dev/shm/ToplingTemp/Topling-199246-rCkBSf 2024/11/23-21:55:22.076332 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282244.sst): first pass time = 3.46's, 155.389'MB/sec 2024/11/23-21:55:26.505114 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282244.sst): old prealloc_size = 590558003, real_size = 107524320 2024/11/23-21:55:26.507243 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282244.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 3709203 prefix = 4 raw-key = 0.0593 GB zip-key = 0.0071 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.4330 GB zip-val = 0.0995 GB avg-val = 116.74 avg-zval = 26.83 2024/11/23-21:55:26.507288 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282244.sst: sumWaitingMem = 0.010 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0102 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:55:29.428749 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282244.sst): second pass time = 4.43's, 97.817'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.92's, 37.945'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3414.132'MB/sec (index lex order gen) rebuild zvType time = 2.42's, 0.383'MB/sec write SST data time = 0.50's, 220.959'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.321'MB, throughput = 130482.034'MB/sec zip my value time = 4.43's, unzip length = 0.433'GB zip my value throughput = 97.817'MB/sec zip pipeline throughput = 97.817'MB/sec entries = 3709203 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.74 avg-zval = 26.83 usrkeys = 3709203 avg-key = 20.00 avg-zkey = 1.91 avg-val = 116.74 avg-zval = 26.83 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0742 GB value = 0.4330 GB dict = 4.32 MB all = 0.5369 GB } __ZipSize{ index = 0.0071 GB value = 0.0995 GB dict = 4.32 MB all = 0.1109 GB } UnZip/Zip{ index = 10.4528 value = 4.3519 dict = 1.00 all = 4.8394 } Zip/UnZip{ index = 0.0957 value = 0.2298 dict = 1.00 all = 0.2066 } ---------------------------- total value len = 11.882525 GB avg = 0.152 KB (by entry num) total key len = 2.165012 GB avg = 0.028 KB total ukey len = 1.515157 GB avg = 0.017 KB total ukey num = 0.091414409 Billion total entry num = 0.078222138 Billion write speed all = 0.300050699 MB/sec (with seq num) write speed all = 0.286684310 MB/sec (without seq num) 2024/11/23-21:55:29.551572 139957273912896 [compaction_job.cc:1948] [default] [JOB 36558] Generated table #282244: 3709203 keys, 110938014 bytes, temperature: kWarm 2024/11/23-21:55:29.551644 139957273912896 EVENT_LOG_v1 {"time_micros": 1732370129551608, "cf_name": "default", "job": 36558, "event": "table_file_creation", "file_number": 282244, "file_size": 110938014, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 99499792, "index_size": 7097024, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 103857684, "raw_average_key_size": 28, "raw_value_size": 433013366, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 3709203, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370118, "oldest_key_time": 0, "file_creation_time": 1732370118, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282244, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:55:29.636194 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282245.sst) /dev/shm/ToplingTemp/Topling-199246-3B1juf 2024/11/23-21:55:33.157532 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282245.sst): first pass time = 3.52's, 152.464'MB/sec 2024/11/23-21:55:37.890594 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282245.sst): old prealloc_size = 590558003, real_size = 110036480 2024/11/23-21:55:37.892709 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282245.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 3701491 prefix = 4 raw-key = 0.0592 GB zip-key = 0.0070 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.4332 GB zip-val = 0.1021 GB avg-val = 117.04 avg-zval = 27.58 2024/11/23-21:55:37.892748 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282245.sst: sumWaitingMem = 0.010 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0102 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:55:42.016798 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282245.sst): second pass time = 4.73's, 91.575'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 4.13's, 27.496'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3372.168'MB/sec (index lex order gen) rebuild zvType time = 3.37's, 0.274'MB/sec write SST data time = 0.75's, 151.259'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.323'MB, throughput = 151375.748'MB/sec zip my value time = 4.73's, unzip length = 0.433'GB zip my value throughput = 91.575'MB/sec zip pipeline throughput = 91.932'MB/sec entries = 3701491 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.04 avg-zval = 27.58 usrkeys = 3701491 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.04 avg-zval = 27.58 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0740 GB value = 0.4332 GB dict = 4.32 MB all = 0.5369 GB } __ZipSize{ index = 0.0070 GB value = 0.1021 GB dict = 4.32 MB all = 0.1135 GB } UnZip/Zip{ index = 10.5202 value = 4.2443 dict = 1.00 all = 4.7321 } Zip/UnZip{ index = 0.0951 value = 0.2356 dict = 1.00 all = 0.2113 } ---------------------------- total value len = 12.315755 GB avg = 0.150 KB (by entry num) total key len = 2.268654 GB avg = 0.028 KB total ukey len = 1.589187 GB avg = 0.017 KB total ukey num = 0.095115900 Billion total entry num = 0.081923629 Billion write speed all = 0.311434345 MB/sec (with seq num) write speed all = 0.297439218 MB/sec (without seq num) 2024/11/23-21:55:42.152678 139957273912896 [compaction_job.cc:1948] [default] [JOB 36558] Generated table #282245: 3701491 keys, 113453898 bytes, temperature: kWarm 2024/11/23-21:55:42.152762 139957273912896 EVENT_LOG_v1 {"time_micros": 1732370142152716, "cf_name": "default", "job": 36558, "event": "table_file_creation", "file_number": 282245, "file_size": 113453898, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 102073968, "index_size": 7036928, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 103641748, "raw_average_key_size": 28, "raw_value_size": 433229182, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 3701491, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370129, "oldest_key_time": 0, "file_creation_time": 1732370129, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282245, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:55:42.340599 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282246.sst) /dev/shm/ToplingTemp/Topling-199246-0MJddh 2024/11/23-21:55:45.846340 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282246.sst): first pass time = 3.51's, 153.143'MB/sec 2024/11/23-21:55:51.017456 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282246.sst): old prealloc_size = 590558003, real_size = 109542816 2024/11/23-21:55:51.019551 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282246.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 3684928 prefix = 4 raw-key = 0.0590 GB zip-key = 0.0070 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.4337 GB zip-val = 0.1016 GB avg-val = 117.69 avg-zval = 27.58 2024/11/23-21:55:51.019592 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282246.sst: sumWaitingMem = 0.010 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0101 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:55:54.115551 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282246.sst): second pass time = 5.17's, 83.907'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 3.10's, 36.464'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3403.061'MB/sec (index lex order gen) rebuild zvType time = 2.59's, 0.355'MB/sec write SST data time = 0.50's, 223.896'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.327'MB, throughput = 133231.901'MB/sec zip my value time = 5.17's, unzip length = 0.434'GB zip my value throughput = 83.907'MB/sec zip pipeline throughput = 104.822'MB/sec entries = 3684928 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.69 avg-zval = 27.58 usrkeys = 3684928 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.69 avg-zval = 27.58 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0737 GB value = 0.4337 GB dict = 4.33 MB all = 0.5369 GB } __ZipSize{ index = 0.0070 GB value = 0.1016 GB dict = 4.33 MB all = 0.1130 GB } UnZip/Zip{ index = 10.5210 value = 4.2679 dict = 1.00 all = 4.7524 } Zip/UnZip{ index = 0.0950 value = 0.2343 dict = 1.00 all = 0.2104 } ---------------------------- total value len = 12.749448 GB avg = 0.149 KB (by entry num) total key len = 2.371832 GB avg = 0.028 KB total ukey len = 1.662886 GB avg = 0.017 KB total ukey num = 0.098800828 Billion total entry num = 0.085608557 Billion write speed all = 0.322815246 MB/sec (with seq num) write speed all = 0.308194395 MB/sec (without seq num) 2024/11/23-21:55:54.204774 139957273912896 [compaction_job.cc:1948] [default] [JOB 36558] Generated table #282246: 3684928 keys, 112967864 bytes, temperature: kWarm 2024/11/23-21:55:54.204839 139957273912896 EVENT_LOG_v1 {"time_micros": 1732370154204807, "cf_name": "default", "job": 36558, "event": "table_file_creation", "file_number": 282246, "file_size": 112967864, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 101616464, "index_size": 7004928, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 103177984, "raw_average_key_size": 28, "raw_value_size": 433693028, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 3684928, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370142, "oldest_key_time": 0, "file_creation_time": 1732370142, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282246, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:55:54.299502 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282247.sst) /dev/shm/ToplingTemp/Topling-199246-cTHFpd 2024/11/23-21:55:57.819220 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282247.sst): first pass time = 3.52's, 152.534'MB/sec 2024/11/23-21:56:02.886093 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282247.sst): old prealloc_size = 590558003, real_size = 109461312 2024/11/23-21:56:02.888668 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282247.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 3684955 prefix = 4 raw-key = 0.0590 GB zip-key = 0.0070 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.4337 GB zip-val = 0.1015 GB avg-val = 117.69 avg-zval = 27.55 2024/11/23-21:56:02.888733 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282247.sst: sumWaitingMem = 0.010 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0101 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:56:06.102737 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282247.sst): second pass time = 5.06's, 85.642'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 3.22's, 35.095'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2780.887'MB/sec (index lex order gen) rebuild zvType time = 2.53's, 0.364'MB/sec write SST data time = 0.69's, 164.440'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.327'MB, throughput = 126938.802'MB/sec zip my value time = 5.06's, unzip length = 0.434'GB zip my value throughput = 85.642'MB/sec zip pipeline throughput = 91.197'MB/sec entries = 3684955 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.69 avg-zval = 27.55 usrkeys = 3684955 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.69 avg-zval = 27.55 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0737 GB value = 0.4337 GB dict = 4.33 MB all = 0.5369 GB } __ZipSize{ index = 0.0070 GB value = 0.1015 GB dict = 4.33 MB all = 0.1129 GB } UnZip/Zip{ index = 10.5197 value = 4.2714 dict = 1.00 all = 4.7558 } Zip/UnZip{ index = 0.0951 value = 0.2341 dict = 1.00 all = 0.2103 } ---------------------------- total value len = 13.183140 GB avg = 0.148 KB (by entry num) total key len = 2.475011 GB avg = 0.028 KB total ukey len = 1.736585 GB avg = 0.017 KB total ukey num = 0.102485783 Billion total entry num = 0.089293512 Billion write speed all = 0.334191065 MB/sec (with seq num) write speed all = 0.318944772 MB/sec (without seq num) 2024/11/23-21:56:06.227973 139957273912896 [compaction_job.cc:1948] [default] [JOB 36558] Generated table #282247: 3684955 keys, 112886522 bytes, temperature: kWarm 2024/11/23-21:56:06.228060 139957273912896 EVENT_LOG_v1 {"time_micros": 1732370166228023, "cf_name": "default", "job": 36558, "event": "table_file_creation", "file_number": 282247, "file_size": 112886522, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 101534064, "index_size": 7005824, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 103178740, "raw_average_key_size": 28, "raw_value_size": 433692242, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 3684955, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370154, "oldest_key_time": 0, "file_creation_time": 1732370154, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282247, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:56:06.316232 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282248.sst) /dev/shm/ToplingTemp/Topling-199246-UmFLDe 2024/11/23-21:56:09.791779 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282248.sst): first pass time = 3.48's, 154.473'MB/sec 2024/11/23-21:56:14.383191 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282248.sst): old prealloc_size = 590558003, real_size = 110667680 2024/11/23-21:56:14.385217 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282248.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 3691902 prefix = 4 raw-key = 0.0591 GB zip-key = 0.0070 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.4335 GB zip-val = 0.1027 GB avg-val = 117.42 avg-zval = 27.82 2024/11/23-21:56:14.385269 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282248.sst: sumWaitingMem = 0.010 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0102 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:56:17.483295 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282248.sst): second pass time = 4.59's, 94.455'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 3.10's, 36.802'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3534.479'MB/sec (index lex order gen) rebuild zvType time = 2.60's, 0.355'MB/sec write SST data time = 0.50's, 228.877'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.325'MB, throughput = 111084.866'MB/sec zip my value time = 4.59's, unzip length = 0.433'GB zip my value throughput = 94.455'MB/sec zip pipeline throughput = 105.756'MB/sec entries = 3691902 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.42 avg-zval = 27.82 usrkeys = 3691902 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.42 avg-zval = 27.82 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0738 GB value = 0.4335 GB dict = 4.32 MB all = 0.5369 GB } __ZipSize{ index = 0.0070 GB value = 0.1027 GB dict = 4.32 MB all = 0.1141 GB } UnZip/Zip{ index = 10.5213 value = 4.2199 dict = 1.00 all = 4.7057 } Zip/UnZip{ index = 0.0950 value = 0.2370 dict = 1.00 all = 0.2125 } ---------------------------- total value len = 13.616638 GB avg = 0.146 KB (by entry num) total key len = 2.578384 GB avg = 0.028 KB total ukey len = 1.810423 GB avg = 0.017 KB total ukey num = 0.106177685 Billion total entry num = 0.092985414 Billion write speed all = 0.345565538 MB/sec (with seq num) write speed all = 0.329692732 MB/sec (without seq num) 2024/11/23-21:56:17.630269 139957273912896 [compaction_job.cc:1948] [default] [JOB 36558] Generated table #282248: 3691902 keys, 114089050 bytes, temperature: kWarm 2024/11/23-21:56:17.630336 139957273912896 EVENT_LOG_v1 {"time_micros": 1732370177630302, "cf_name": "default", "job": 36558, "event": "table_file_creation", "file_number": 282248, "file_size": 114089050, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 102726544, "index_size": 7017984, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 103373256, "raw_average_key_size": 28, "raw_value_size": 433497768, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 3691902, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370166, "oldest_key_time": 0, "file_creation_time": 1732370166, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282248, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:56:17.716417 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282249.sst) /dev/shm/ToplingTemp/Topling-199246-MPGgxh 2024/11/23-21:56:21.173626 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282249.sst): first pass time = 3.46's, 155.292'MB/sec 2024/11/23-21:56:25.468436 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282249.sst): old prealloc_size = 590558003, real_size = 110142656 2024/11/23-21:56:25.470422 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282249.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 3684685 prefix = 4 raw-key = 0.0590 GB zip-key = 0.0070 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.4337 GB zip-val = 0.1022 GB avg-val = 117.70 avg-zval = 27.74 2024/11/23-21:56:25.470465 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282249.sst: sumWaitingMem = 0.010 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0101 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:56:28.368857 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282249.sst): second pass time = 4.29's, 101.029'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.90's, 39.156'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3611.329'MB/sec (index lex order gen) rebuild zvType time = 2.40's, 0.384'MB/sec write SST data time = 0.50's, 227.548'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.326'MB, throughput = 135578.766'MB/sec zip my value time = 4.29's, unzip length = 0.434'GB zip my value throughput = 101.029'MB/sec zip pipeline throughput = 101.029'MB/sec entries = 3684685 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.70 avg-zval = 27.74 usrkeys = 3684685 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.70 avg-zval = 27.74 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0737 GB value = 0.4337 GB dict = 4.33 MB all = 0.5369 GB } __ZipSize{ index = 0.0070 GB value = 0.1022 GB dict = 4.33 MB all = 0.1136 GB } UnZip/Zip{ index = 10.5199 value = 4.2430 dict = 1.00 all = 4.7273 } Zip/UnZip{ index = 0.0951 value = 0.2357 dict = 1.00 all = 0.2115 } ---------------------------- total value len = 14.050338 GB avg = 0.145 KB (by entry num) total key len = 2.681555 GB avg = 0.028 KB total ukey len = 1.884117 GB avg = 0.017 KB total ukey num = 0.109862370 Billion total entry num = 0.096670099 Billion write speed all = 0.356938258 MB/sec (with seq num) write speed all = 0.340440300 MB/sec (without seq num) 2024/11/23-21:56:28.500851 139957273912896 [compaction_job.cc:1948] [default] [JOB 36558] Generated table #282249: 3684685 keys, 113567548 bytes, temperature: kWarm 2024/11/23-21:56:28.501376 139957273912896 EVENT_LOG_v1 {"time_micros": 1732370188500885, "cf_name": "default", "job": 36558, "event": "table_file_creation", "file_number": 282249, "file_size": 113567548, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 102216112, "index_size": 7005184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 103171180, "raw_average_key_size": 28, "raw_value_size": 433699850, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 3684685, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370177, "oldest_key_time": 0, "file_creation_time": 1732370177, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282249, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:56:28.591660 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282250.sst) /dev/shm/ToplingTemp/Topling-199246-aNjife 2024/11/23-21:56:32.081896 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282250.sst): first pass time = 3.49's, 153.823'MB/sec 2024/11/23-21:56:36.524954 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282250.sst): old prealloc_size = 590558003, real_size = 110034160 2024/11/23-21:56:36.527071 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282250.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 3693109 prefix = 4 raw-key = 0.0591 GB zip-key = 0.0070 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.4335 GB zip-val = 0.1021 GB avg-val = 117.37 avg-zval = 27.66 2024/11/23-21:56:36.527112 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282250.sst: sumWaitingMem = 0.010 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0102 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:56:39.574534 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282250.sst): second pass time = 4.44's, 97.609'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 3.05's, 37.209'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3389.116'MB/sec (index lex order gen) rebuild zvType time = 2.55's, 0.362'MB/sec write SST data time = 0.50's, 227.089'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.325'MB, throughput = 86555.137'MB/sec zip my value time = 4.44's, unzip length = 0.433'GB zip my value throughput = 97.609'MB/sec zip pipeline throughput = 97.609'MB/sec entries = 3693109 avg-key = 28.00 avg-zkey = 1.89 avg-val = 117.37 avg-zval = 27.66 usrkeys = 3693109 avg-key = 20.00 avg-zkey = 1.89 avg-val = 117.37 avg-zval = 27.66 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0739 GB value = 0.4335 GB dict = 4.32 MB all = 0.5369 GB } __ZipSize{ index = 0.0070 GB value = 0.1021 GB dict = 4.32 MB all = 0.1135 GB } UnZip/Zip{ index = 10.5899 value = 4.2440 dict = 1.00 all = 4.7314 } Zip/UnZip{ index = 0.0944 value = 0.2356 dict = 1.00 all = 0.2114 } ---------------------------- total value len = 14.483802 GB avg = 0.144 KB (by entry num) total key len = 2.784962 GB avg = 0.028 KB total ukey len = 1.957979 GB avg = 0.017 KB total ukey num = 0.113555479 Billion total entry num = 0.100363208 Billion write speed all = 0.368303181 MB/sec (with seq num) write speed all = 0.351179042 MB/sec (without seq num) 2024/11/23-21:56:39.704736 139957273912896 [compaction_job.cc:1948] [default] [JOB 36558] Generated table #282250: 3693109 keys, 113470100 bytes, temperature: kWarm 2024/11/23-21:56:39.704804 139957273912896 EVENT_LOG_v1 {"time_micros": 1732370199704770, "cf_name": "default", "job": 36558, "event": "table_file_creation", "file_number": 282250, "file_size": 113470100, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 102135904, "index_size": 6974784, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 103407052, "raw_average_key_size": 28, "raw_value_size": 433463954, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 3693109, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370188, "oldest_key_time": 0, "file_creation_time": 1732370188, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282250, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:56:39.793652 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282251.sst) /dev/shm/ToplingTemp/Topling-199246-Bf70kh 2024/11/23-21:56:40.349700 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282251.sst): first pass time = 0.56's, 119.757'MB/sec 2024/11/23-21:56:41.005653 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282251.sst): old prealloc_size = 590558003, real_size = 18185056 2024/11/23-21:56:41.005972 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282251.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 460008 prefix = 4 raw-key = 0.0074 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.0537 GB zip-val = 0.0172 GB avg-val = 116.75 avg-zval = 37.39 2024/11/23-21:56:41.006032 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282251.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.067 GB 2024/11/23-21:56:41.351077 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282251.sst): second pass time = 0.66's, 81.966'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.35's, 53.901'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2760.510'MB/sec (index lex order gen) rebuild zvType time = 0.28's, 0.407'MB/sec write SST data time = 0.06's, 298.016'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.542'MB, throughput = 17733.530'MB/sec zip my value time = 0.66's, unzip length = 0.054'GB zip my value throughput = 81.966'MB/sec zip pipeline throughput = 81.966'MB/sec entries = 460008 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.75 avg-zval = 37.39 usrkeys = 460008 avg-key = 20.00 avg-zkey = 1.89 avg-val = 116.75 avg-zval = 37.39 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0092 GB value = 0.0537 GB dict = 0.54 MB all = 0.0666 GB } __ZipSize{ index = 0.0009 GB value = 0.0172 GB dict = 0.54 MB all = 0.0186 GB } UnZip/Zip{ index = 10.5747 value = 3.1225 dict = 1.00 all = 3.5766 } Zip/UnZip{ index = 0.0946 value = 0.3203 dict = 1.00 all = 0.2796 } ---------------------------- total value len = 14.537508 GB avg = 0.144 KB (by entry num) total key len = 2.797843 GB avg = 0.028 KB total ukey len = 1.967179 GB avg = 0.017 KB total ukey num = 0.114015487 Billion total entry num = 0.100823216 Billion write speed all = 0.369709307 MB/sec (with seq num) write speed all = 0.352507332 MB/sec (without seq num) 2024/11/23-21:56:41.378786 139957273912896 [compaction_job.cc:1948] [default] [JOB 36558] Generated table #282251: 460008 keys, 18617380 bytes, temperature: kWarm 2024/11/23-21:56:41.378852 139957273912896 EVENT_LOG_v1 {"time_micros": 1732370201378819, "cf_name": "default", "job": 36558, "event": "table_file_creation", "file_number": 282251, "file_size": 18617380, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 17199824, "index_size": 870016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 12880224, "raw_average_key_size": 28, "raw_value_size": 53706076, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 460008, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370199, "oldest_key_time": 0, "file_creation_time": 1732370199, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282251, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:56:41.408591 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282252.sst) /dev/shm/ToplingTemp/Topling-199246-Dom3wf 2024/11/23-21:56:42.320072 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282252.sst): first pass time = 0.91's, 9.107'MB/sec 2024/11/23-21:56:42.475810 139957273912896 [top_zip_table_builder.cc:2726] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282252.sst): part-0: reject ZipStore ratio = 1.032 2024/11/23-21:56:42.534661 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282252.sst): old prealloc_size = 590558003, real_size = 6796480 2024/11/23-21:56:42.534694 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282252.sst): index type = UintIndex_AllOne , store type = ZipOffsetBlobStore usrkeys = 100000 prefix = 4 raw-key = 0.0004 GB zip-key = 0.0000 GB avg-key = 4.00 avg-zkey = 0.00 raw-val = 0.0067 GB zip-val = 0.0068 GB avg-val = 67.01 avg-zval = 67.71 2024/11/23-21:56:42.554525 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282252.sst): second pass time = 0.16's, 43.204'MB/sec, value only(80.7% of KV) wait indexing time = 0.06's, remap KeyValue time = 0.02's, 341.177'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 8.962'MB/sec (index lex order gen) rebuild zvType time = 0.01's, 2.067'MB/sec write SST data time = 0.01's, 875.405'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 0.16's, unzip length = 0.007'GB zip my value throughput = 43.204'MB/sec zip pipeline throughput = 43.204'MB/sec entries = 100000 avg-key = 16.00 avg-zkey = 0.00 avg-val = 67.01 avg-zval = 67.71 usrkeys = 100000 avg-key = 8.00 avg-zkey = 0.00 avg-val = 67.01 avg-zval = 67.71 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0008 GB value = 0.0067 GB dict = 0.00 MB all = 0.0083 GB } __ZipSize{ index = 0.0000 GB value = 0.0068 GB dict = 0.00 MB all = 0.0068 GB } UnZip/Zip{ index =4166.6667 value = 0.9896 dict = 0.00 all = 1.2257 } Zip/UnZip{ index = 0.0002 value = 1.0105 dict = 0.00 all = 0.8159 } ---------------------------- total value len = 14.544208 GB avg = 0.144 KB (by entry num) total key len = 2.799443 GB avg = 0.028 KB total ukey len = 1.967979 GB avg = 0.017 KB total ukey num = 0.114115487 Billion total entry num = 0.100923216 Billion write speed all = 0.369876842 MB/sec (with seq num) write speed all = 0.352658248 MB/sec (without seq num) 2024/11/23-21:56:42.568630 139957273912896 [compaction_job.cc:1948] [default] [JOB 36558] Generated table #282252: 100000 keys, 6772487 bytes, temperature: kWarm 2024/11/23-21:56:42.568680 139957273912896 EVENT_LOG_v1 {"time_micros": 1732370202568652, "cf_name": "default", "job": 36558, "event": "table_file_creation", "file_number": 282252, "file_size": 6772487, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 6771152, "index_size": 192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 1600000, "raw_average_key_size": 16, "raw_value_size": 6700718, "raw_average_value_size": 67, "num_data_blocks": 1, "num_entries": 100000, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-UintIndex_AllOne-ZipOffsetBlobStore", "creation_time": 1732370201, "oldest_key_time": 0, "file_creation_time": 1732370201, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282252, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:56:42.591115 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282253.sst) /dev/shm/ToplingTemp/Topling-199246-nTxlRf 2024/11/23-21:56:45.346841 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282253.sst): first pass time = 2.76's, 194.823'MB/sec 2024/11/23-21:56:49.917877 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282253.sst): old prealloc_size = 590558003, real_size = 165158736 2024/11/23-21:56:49.918222 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282253.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517709 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1642 GB avg-val =1017.01 avg-zval = 317.09 2024/11/23-21:56:49.918263 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282253.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:56:50.560183 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282253.sst): second pass time = 4.57's, 115.242'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 265.156'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2868.004'MB/sec (index lex order gen) rebuild zvType time = 0.46's, 0.278'MB/sec write SST data time = 0.18's, 962.053'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.273'MB, throughput = 159107.550'MB/sec zip my value time = 4.57's, unzip length = 0.527'GB zip my value throughput = 115.242'MB/sec zip pipeline throughput = 115.242'MB/sec entries = 517709 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.01 avg-zval = 317.09 usrkeys = 517709 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.01 avg-zval = 317.09 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1642 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1365 value = 3.2074 dict = 1.00 all = 3.1524 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 15.070726 GB avg = 0.149 KB (by entry num) total key len = 2.809797 GB avg = 0.028 KB total ukey len = 1.974192 GB avg = 0.017 KB total ukey num = 0.114633196 Billion total entry num = 0.101440925 Billion write speed all = 0.381261258 MB/sec (with seq num) write speed all = 0.363957292 MB/sec (without seq num) 2024/11/23-21:56:50.746457 139957273912896 [compaction_job.cc:1948] [default] [JOB 36558] Generated table #282253: 517709 keys, 170305006 bytes, temperature: kWarm 2024/11/23-21:56:50.746524 139957273912896 EVENT_LOG_v1 {"time_micros": 1732370210746489, "cf_name": "default", "job": 36558, "event": "table_file_creation", "file_number": 282253, "file_size": 170305006, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164158592, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354180, "raw_average_key_size": 20, "raw_value_size": 526517224, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 517709, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370202, "oldest_key_time": 0, "file_creation_time": 1732370202, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282253, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:56:50.839922 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282254.sst) /dev/shm/ToplingTemp/Topling-199246-Xsjwrg 2024/11/23-21:56:52.365426 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282254.sst): first pass time = 1.53's, 351.940'MB/sec 2024/11/23-21:56:56.844623 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282254.sst): old prealloc_size = 590558003, real_size = 165155952 2024/11/23-21:56:56.844958 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282254.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517711 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1642 GB avg-val =1017.01 avg-zval = 317.08 2024/11/23-21:56:56.845022 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282254.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:56:57.492865 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282254.sst): second pass time = 4.48's, 117.611'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 262.720'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2948.324'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.276'MB/sec write SST data time = 0.18's, 951.773'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.272'MB, throughput = 150713.763'MB/sec zip my value time = 4.48's, unzip length = 0.527'GB zip my value throughput = 117.611'MB/sec zip pipeline throughput = 117.611'MB/sec entries = 517711 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.01 avg-zval = 317.08 usrkeys = 517711 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.01 avg-zval = 317.08 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1642 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1365 value = 3.2074 dict = 1.00 all = 3.1525 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 15.597242 GB avg = 0.153 KB (by entry num) total key len = 2.820151 GB avg = 0.028 KB total ukey len = 1.980404 GB avg = 0.017 KB total ukey num = 0.115150907 Billion total entry num = 0.101958636 Billion write speed all = 0.392650764 MB/sec (with seq num) write speed all = 0.375261056 MB/sec (without seq num) 2024/11/23-21:56:57.680542 139957273912896 [compaction_job.cc:1948] [default] [JOB 36558] Generated table #282254: 517711 keys, 170301414 bytes, temperature: kWarm 2024/11/23-21:56:57.680613 139957273912896 EVENT_LOG_v1 {"time_micros": 1732370217680578, "cf_name": "default", "job": 36558, "event": "table_file_creation", "file_number": 282254, "file_size": 170301414, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164155808, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354220, "raw_average_key_size": 20, "raw_value_size": 526516872, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 517711, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370210, "oldest_key_time": 0, "file_creation_time": 1732370210, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282254, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:56:57.785057 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282255.sst) /dev/shm/ToplingTemp/Topling-199246-t1ydkh 2024/11/23-21:57:00.599435 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282255.sst): first pass time = 2.81's, 190.763'MB/sec 2024/11/23-21:57:05.172305 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282255.sst): old prealloc_size = 590558003, real_size = 165164032 2024/11/23-21:57:05.172636 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282255.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517709 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1642 GB avg-val =1017.01 avg-zval = 317.10 2024/11/23-21:57:05.172675 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282255.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:57:05.817338 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282255.sst): second pass time = 4.57's, 115.201'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 264.043'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3019.668'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.277'MB/sec write SST data time = 0.18's, 960.949'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.272'MB, throughput = 159148.666'MB/sec zip my value time = 4.57's, unzip length = 0.527'GB zip my value throughput = 115.201'MB/sec zip pipeline throughput = 115.201'MB/sec entries = 517709 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.01 avg-zval = 317.10 usrkeys = 517709 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.01 avg-zval = 317.10 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1642 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1365 value = 3.2073 dict = 1.00 all = 3.1523 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 16.123759 GB avg = 0.157 KB (by entry num) total key len = 2.830505 GB avg = 0.028 KB total ukey len = 1.986617 GB avg = 0.017 KB total ukey num = 0.115668616 Billion total entry num = 0.102476345 Billion write speed all = 0.404024918 MB/sec (with seq num) write speed all = 0.386550012 MB/sec (without seq num) 2024/11/23-21:57:06.002436 139957273912896 [compaction_job.cc:1948] [default] [JOB 36558] Generated table #282255: 517709 keys, 170310089 bytes, temperature: kWarm 2024/11/23-21:57:06.002509 139957273912896 EVENT_LOG_v1 {"time_micros": 1732370226002473, "cf_name": "default", "job": 36558, "event": "table_file_creation", "file_number": 282255, "file_size": 170310089, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164163888, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354180, "raw_average_key_size": 20, "raw_value_size": 526516957, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 517709, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370217, "oldest_key_time": 0, "file_creation_time": 1732370217, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282255, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:57:06.110206 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282256.sst) /dev/shm/ToplingTemp/Topling-199246-tTZvee 2024/11/23-21:57:08.636865 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282256.sst): first pass time = 2.53's, 212.486'MB/sec 2024/11/23-21:57:13.196018 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282256.sst): old prealloc_size = 590558003, real_size = 165157760 2024/11/23-21:57:13.196372 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282256.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517719 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1642 GB avg-val =1016.99 avg-zval = 317.08 2024/11/23-21:57:13.196415 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282256.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:57:13.842792 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282256.sst): second pass time = 4.56's, 115.550'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 263.313'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2871.798'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.275'MB/sec write SST data time = 0.18's, 968.010'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.272'MB, throughput = 149999.858'MB/sec zip my value time = 4.56's, unzip length = 0.527'GB zip my value throughput = 115.550'MB/sec zip pipeline throughput = 115.550'MB/sec entries = 517719 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1016.99 avg-zval = 317.08 usrkeys = 517719 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1016.99 avg-zval = 317.08 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1642 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1366 value = 3.2074 dict = 1.00 all = 3.1524 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 16.650276 GB avg = 0.162 KB (by entry num) total key len = 2.840859 GB avg = 0.028 KB total ukey len = 1.992829 GB avg = 0.017 KB total ukey num = 0.116186335 Billion total entry num = 0.102994064 Billion write speed all = 0.415397687 MB/sec (with seq num) write speed all = 0.397837501 MB/sec (without seq num) 2024/11/23-21:57:14.028557 139957273912896 [compaction_job.cc:1948] [default] [JOB 36558] Generated table #282256: 517719 keys, 170304035 bytes, temperature: kWarm 2024/11/23-21:57:14.028626 139957273912896 EVENT_LOG_v1 {"time_micros": 1732370234028590, "cf_name": "default", "job": 36558, "event": "table_file_creation", "file_number": 282256, "file_size": 170304035, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164157616, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354380, "raw_average_key_size": 20, "raw_value_size": 526516945, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 517719, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370226, "oldest_key_time": 0, "file_creation_time": 1732370226, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282256, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:57:14.129847 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282257.sst) /dev/shm/ToplingTemp/Topling-199246-FabVof 2024/11/23-21:57:16.862228 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282257.sst): first pass time = 2.73's, 196.488'MB/sec 2024/11/23-21:57:21.518843 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282257.sst): old prealloc_size = 590558003, real_size = 165573008 2024/11/23-21:57:21.519202 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282257.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517715 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1646 GB avg-val =1017.00 avg-zval = 317.88 2024/11/23-21:57:21.519242 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282257.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:57:22.161355 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282257.sst): second pass time = 4.65's, 113.126'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 265.715'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2716.097'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.277'MB/sec write SST data time = 0.17's, 979.709'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.273'MB, throughput = 158456.966'MB/sec zip my value time = 4.65's, unzip length = 0.527'GB zip my value throughput = 113.126'MB/sec zip pipeline throughput = 113.126'MB/sec entries = 517715 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 317.88 usrkeys = 517715 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 317.88 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1646 GB dict = 5.27 MB all = 0.1707 GB } UnZip/Zip{ index = 7.1366 value = 3.1993 dict = 1.00 all = 3.1448 } Zip/UnZip{ index = 0.1401 value = 0.3126 dict = 1.00 all = 0.3180 } ---------------------------- total value len = 17.176793 GB avg = 0.166 KB (by entry num) total key len = 2.851214 GB avg = 0.028 KB total ukey len = 1.999042 GB avg = 0.017 KB total ukey num = 0.116704050 Billion total entry num = 0.103511779 Billion write speed all = 0.426763901 MB/sec (with seq num) write speed all = 0.409118574 MB/sec (without seq num) 2024/11/23-21:57:22.379260 139957273912896 [compaction_job.cc:1948] [default] [JOB 36558] Generated table #282257: 517715 keys, 170719586 bytes, temperature: kWarm 2024/11/23-21:57:22.379332 139957273912896 EVENT_LOG_v1 {"time_micros": 1732370242379295, "cf_name": "default", "job": 36558, "event": "table_file_creation", "file_number": 282257, "file_size": 170719586, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164572864, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354300, "raw_average_key_size": 20, "raw_value_size": 526517034, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 517715, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370234, "oldest_key_time": 0, "file_creation_time": 1732370234, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282257, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:57:22.480886 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282258.sst) /dev/shm/ToplingTemp/Topling-199246-7ResPd 2024/11/23-21:57:25.170365 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282258.sst): first pass time = 2.69's, 199.623'MB/sec 2024/11/23-21:57:29.667578 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282258.sst): old prealloc_size = 590558003, real_size = 165165680 2024/11/23-21:57:29.667890 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282258.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517711 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1642 GB avg-val =1017.01 avg-zval = 317.10 2024/11/23-21:57:29.667937 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282258.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:57:30.321832 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282258.sst): second pass time = 4.50's, 117.128'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 260.323'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3261.981'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.275'MB/sec write SST data time = 0.18's, 928.047'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.272'MB, throughput = 158003.656'MB/sec zip my value time = 4.50's, unzip length = 0.527'GB zip my value throughput = 117.128'MB/sec zip pipeline throughput = 117.128'MB/sec entries = 517711 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.01 avg-zval = 317.10 usrkeys = 517711 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.01 avg-zval = 317.10 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1642 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1365 value = 3.2072 dict = 1.00 all = 3.1523 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 17.703311 GB avg = 0.170 KB (by entry num) total key len = 2.861568 GB avg = 0.028 KB total ukey len = 2.005254 GB avg = 0.017 KB total ukey num = 0.117221761 Billion total entry num = 0.104029490 Billion write speed all = 0.438127570 MB/sec (with seq num) write speed all = 0.420397074 MB/sec (without seq num) 2024/11/23-21:57:30.512932 139957273912896 [compaction_job.cc:1948] [default] [JOB 36558] Generated table #282258: 517711 keys, 170311584 bytes, temperature: kWarm 2024/11/23-21:57:30.513028 139957273912896 EVENT_LOG_v1 {"time_micros": 1732370250512968, "cf_name": "default", "job": 36558, "event": "table_file_creation", "file_number": 282258, "file_size": 170311584, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164165536, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354220, "raw_average_key_size": 20, "raw_value_size": 526517499, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 517711, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370242, "oldest_key_time": 0, "file_creation_time": 1732370242, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282258, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:57:30.607500 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282259.sst) /dev/shm/ToplingTemp/Topling-199246-UXHCWf 2024/11/23-21:57:33.161896 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282259.sst): first pass time = 2.55's, 210.179'MB/sec 2024/11/23-21:57:37.632123 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282259.sst): old prealloc_size = 590558003, real_size = 165152384 2024/11/23-21:57:37.632444 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282259.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517716 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1642 GB avg-val =1017.00 avg-zval = 317.07 2024/11/23-21:57:37.632484 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282259.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:57:38.283679 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282259.sst): second pass time = 4.47's, 117.840'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 261.379'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3236.789'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.274'MB/sec write SST data time = 0.18's, 952.361'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.272'MB, throughput = 156159.903'MB/sec zip my value time = 4.47's, unzip length = 0.527'GB zip my value throughput = 117.840'MB/sec zip pipeline throughput = 117.840'MB/sec entries = 517716 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 317.07 usrkeys = 517716 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 317.07 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1642 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1366 value = 3.2075 dict = 1.00 all = 3.1525 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 18.229828 GB avg = 0.174 KB (by entry num) total key len = 2.871922 GB avg = 0.027 KB total ukey len = 2.011467 GB avg = 0.017 KB total ukey num = 0.117739477 Billion total entry num = 0.104547206 Billion write speed all = 0.449489184 MB/sec (with seq num) write speed all = 0.431673472 MB/sec (without seq num) 2024/11/23-21:57:38.471832 139957273912896 [compaction_job.cc:1948] [default] [JOB 36558] Generated table #282259: 517716 keys, 170297810 bytes, temperature: kWarm 2024/11/23-21:57:38.471906 139957273912896 EVENT_LOG_v1 {"time_micros": 1732370258471869, "cf_name": "default", "job": 36558, "event": "table_file_creation", "file_number": 282259, "file_size": 170297810, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164152240, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354320, "raw_average_key_size": 20, "raw_value_size": 526517114, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 517716, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370250, "oldest_key_time": 0, "file_creation_time": 1732370250, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282259, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:57:38.564729 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282260.sst) /dev/shm/ToplingTemp/Topling-199246-2wQ6mf 2024/11/23-21:57:41.236674 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282260.sst): first pass time = 2.67's, 200.933'MB/sec 2024/11/23-21:57:45.808469 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282260.sst): old prealloc_size = 590558003, real_size = 165156096 2024/11/23-21:57:45.808782 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282260.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517720 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1642 GB avg-val =1016.99 avg-zval = 317.07 2024/11/23-21:57:45.808822 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282260.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:57:46.460358 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282260.sst): second pass time = 4.57's, 115.219'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 261.253'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3214.510'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.274'MB/sec write SST data time = 0.18's, 946.348'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.272'MB, throughput = 152567.224'MB/sec zip my value time = 4.57's, unzip length = 0.527'GB zip my value throughput = 115.219'MB/sec zip pipeline throughput = 115.219'MB/sec entries = 517720 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1016.99 avg-zval = 317.07 usrkeys = 517720 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1016.99 avg-zval = 317.07 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1642 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1366 value = 3.2074 dict = 1.00 all = 3.1525 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 18.756345 GB avg = 0.179 KB (by entry num) total key len = 2.882277 GB avg = 0.027 KB total ukey len = 2.017680 GB avg = 0.017 KB total ukey num = 0.118257197 Billion total entry num = 0.105064926 Billion write speed all = 0.460844838 MB/sec (with seq num) write speed all = 0.442944020 MB/sec (without seq num) 2024/11/23-21:57:46.645451 139957273912896 [compaction_job.cc:1948] [default] [JOB 36558] Generated table #282260: 517720 keys, 170301989 bytes, temperature: kWarm 2024/11/23-21:57:46.645521 139957273912896 EVENT_LOG_v1 {"time_micros": 1732370266645485, "cf_name": "default", "job": 36558, "event": "table_file_creation", "file_number": 282260, "file_size": 170301989, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164155952, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354400, "raw_average_key_size": 20, "raw_value_size": 526517100, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 517720, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370258, "oldest_key_time": 0, "file_creation_time": 1732370258, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282260, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:57:46.737554 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282261.sst) /dev/shm/ToplingTemp/Topling-199246-cJNzBe 2024/11/23-21:57:49.410881 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282261.sst): first pass time = 2.67's, 200.829'MB/sec 2024/11/23-21:57:53.927534 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282261.sst): old prealloc_size = 590558003, real_size = 165151696 2024/11/23-21:57:53.927864 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282261.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517722 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1642 GB avg-val =1016.99 avg-zval = 317.07 2024/11/23-21:57:53.927905 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282261.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:57:54.574015 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282261.sst): second pass time = 4.51's, 116.617'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 263.433'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3025.882'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.277'MB/sec write SST data time = 0.18's, 951.360'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.272'MB, throughput = 149178.466'MB/sec zip my value time = 4.51's, unzip length = 0.527'GB zip my value throughput = 116.617'MB/sec zip pipeline throughput = 116.617'MB/sec entries = 517722 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1016.99 avg-zval = 317.07 usrkeys = 517722 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1016.99 avg-zval = 317.07 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1642 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1367 value = 3.2075 dict = 1.00 all = 3.1525 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 19.282862 GB avg = 0.183 KB (by entry num) total key len = 2.892631 GB avg = 0.027 KB total ukey len = 2.023892 GB avg = 0.017 KB total ukey num = 0.118774919 Billion total entry num = 0.105582648 Billion write speed all = 0.472197164 MB/sec (with seq num) write speed all = 0.454211245 MB/sec (without seq num) 2024/11/23-21:57:54.765270 139957273912896 [compaction_job.cc:1948] [default] [JOB 36558] Generated table #282261: 517722 keys, 170297427 bytes, temperature: kWarm 2024/11/23-21:57:54.765345 139957273912896 EVENT_LOG_v1 {"time_micros": 1732370274765308, "cf_name": "default", "job": 36558, "event": "table_file_creation", "file_number": 282261, "file_size": 170297427, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164151552, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354440, "raw_average_key_size": 20, "raw_value_size": 526516680, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 517722, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370266, "oldest_key_time": 0, "file_creation_time": 1732370266, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282261, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:57:54.857373 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282262.sst) /dev/shm/ToplingTemp/Topling-199246-BdVGEf 2024/11/23-21:57:57.470579 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282262.sst): first pass time = 2.61's, 205.449'MB/sec 2024/11/23-21:58:02.035350 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282262.sst): old prealloc_size = 590558003, real_size = 165155232 2024/11/23-21:58:02.035652 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282262.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517721 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1642 GB avg-val =1016.99 avg-zval = 317.07 2024/11/23-21:58:02.035694 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282262.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:58:02.689805 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282262.sst): second pass time = 4.56's, 115.391'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 260.227'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3388.930'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.275'MB/sec write SST data time = 0.18's, 928.191'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.272'MB, throughput = 144320.103'MB/sec zip my value time = 4.56's, unzip length = 0.527'GB zip my value throughput = 115.391'MB/sec zip pipeline throughput = 115.391'MB/sec entries = 517721 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1016.99 avg-zval = 317.07 usrkeys = 517721 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1016.99 avg-zval = 317.07 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1642 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1366 value = 3.2074 dict = 1.00 all = 3.1525 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 19.809378 GB avg = 0.187 KB (by entry num) total key len = 2.902986 GB avg = 0.027 KB total ukey len = 2.030105 GB avg = 0.017 KB total ukey num = 0.119292640 Billion total entry num = 0.106100369 Billion write speed all = 0.483545542 MB/sec (with seq num) write speed all = 0.465474553 MB/sec (without seq num) 2024/11/23-21:58:02.880194 139957273912896 [compaction_job.cc:1948] [default] [JOB 36558] Generated table #282262: 517721 keys, 170301282 bytes, temperature: kWarm 2024/11/23-21:58:02.880266 139957273912896 EVENT_LOG_v1 {"time_micros": 1732370282880229, "cf_name": "default", "job": 36558, "event": "table_file_creation", "file_number": 282262, "file_size": 170301282, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164155088, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354420, "raw_average_key_size": 20, "raw_value_size": 526516542, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 517721, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370274, "oldest_key_time": 0, "file_creation_time": 1732370274, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282262, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:58:02.977872 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282263.sst) /dev/shm/ToplingTemp/Topling-199246-m5IaTd 2024/11/23-21:58:05.569628 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282263.sst): first pass time = 2.59's, 207.149'MB/sec 2024/11/23-21:58:10.133260 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282263.sst): old prealloc_size = 590558003, real_size = 165155264 2024/11/23-21:58:10.133563 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282263.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517712 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1642 GB avg-val =1017.01 avg-zval = 317.08 2024/11/23-21:58:10.133606 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282263.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:58:10.784065 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282263.sst): second pass time = 4.56's, 115.417'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 261.688'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3424.727'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.275'MB/sec write SST data time = 0.18's, 945.373'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.272'MB, throughput = 140545.442'MB/sec zip my value time = 4.56's, unzip length = 0.527'GB zip my value throughput = 115.417'MB/sec zip pipeline throughput = 115.417'MB/sec entries = 517712 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.01 avg-zval = 317.08 usrkeys = 517712 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.01 avg-zval = 317.08 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1642 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1365 value = 3.2074 dict = 1.00 all = 3.1525 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 20.335895 GB avg = 0.191 KB (by entry num) total key len = 2.913340 GB avg = 0.027 KB total ukey len = 2.036317 GB avg = 0.017 KB total ukey num = 0.119810352 Billion total entry num = 0.106618081 Billion write speed all = 0.494890225 MB/sec (with seq num) write speed all = 0.476734188 MB/sec (without seq num) 2024/11/23-21:58:10.974961 139957273912896 [compaction_job.cc:1948] [default] [JOB 36558] Generated table #282263: 517712 keys, 170300763 bytes, temperature: kWarm 2024/11/23-21:58:10.975049 139957273912896 EVENT_LOG_v1 {"time_micros": 1732370290975012, "cf_name": "default", "job": 36558, "event": "table_file_creation", "file_number": 282263, "file_size": 170300763, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164155120, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354240, "raw_average_key_size": 20, "raw_value_size": 526516690, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 517712, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370282, "oldest_key_time": 0, "file_creation_time": 1732370282, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282263, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:58:11.072117 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282264.sst) /dev/shm/ToplingTemp/Topling-199246-6GSYFh 2024/11/23-21:58:13.673853 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282264.sst): first pass time = 2.60's, 206.355'MB/sec 2024/11/23-21:58:18.220457 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282264.sst): old prealloc_size = 590558003, real_size = 165152112 2024/11/23-21:58:18.220776 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282264.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517713 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1642 GB avg-val =1017.01 avg-zval = 317.07 2024/11/23-21:58:18.220817 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282264.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:58:18.877599 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282264.sst): second pass time = 4.54's, 115.857'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.66's, 259.162'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3209.497'MB/sec (index lex order gen) rebuild zvType time = 0.48's, 0.271'MB/sec write SST data time = 0.18's, 947.824'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.273'MB, throughput = 152886.466'MB/sec zip my value time = 4.54's, unzip length = 0.527'GB zip my value throughput = 115.857'MB/sec zip pipeline throughput = 115.857'MB/sec entries = 517713 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.01 avg-zval = 317.07 usrkeys = 517713 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.01 avg-zval = 317.07 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1642 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1365 value = 3.2075 dict = 1.00 all = 3.1525 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 20.862412 GB avg = 0.195 KB (by entry num) total key len = 2.923694 GB avg = 0.027 KB total ukey len = 2.042530 GB avg = 0.017 KB total ukey num = 0.120328065 Billion total entry num = 0.107135794 Billion write speed all = 0.506231012 MB/sec (with seq num) write speed all = 0.487989956 MB/sec (without seq num) 2024/11/23-21:58:19.067604 139957273912896 [compaction_job.cc:1948] [default] [JOB 36558] Generated table #282264: 517713 keys, 170299268 bytes, temperature: kWarm 2024/11/23-21:58:19.067673 139957273912896 EVENT_LOG_v1 {"time_micros": 1732370299067637, "cf_name": "default", "job": 36558, "event": "table_file_creation", "file_number": 282264, "file_size": 170299268, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164151968, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354260, "raw_average_key_size": 20, "raw_value_size": 526516884, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 517713, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370291, "oldest_key_time": 0, "file_creation_time": 1732370291, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282264, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:58:19.166276 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282265.sst) /dev/shm/ToplingTemp/Topling-199246-rv4Ord 2024/11/23-21:58:21.840023 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282265.sst): first pass time = 2.67's, 200.797'MB/sec 2024/11/23-21:58:26.445690 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282265.sst): old prealloc_size = 590558003, real_size = 165581280 2024/11/23-21:58:26.446030 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282265.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517718 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1646 GB avg-val =1017.00 avg-zval = 317.90 2024/11/23-21:58:26.446072 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282265.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:58:27.091931 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282265.sst): second pass time = 4.60's, 114.377'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 264.195'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2920.549'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.277'MB/sec write SST data time = 0.18's, 959.422'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.273'MB, throughput = 152514.680'MB/sec zip my value time = 4.60's, unzip length = 0.527'GB zip my value throughput = 114.377'MB/sec zip pipeline throughput = 114.377'MB/sec entries = 517718 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 317.90 usrkeys = 517718 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 317.90 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1646 GB dict = 5.27 MB all = 0.1707 GB } UnZip/Zip{ index = 7.1366 value = 3.1991 dict = 1.00 all = 3.1446 } Zip/UnZip{ index = 0.1401 value = 0.3126 dict = 1.00 all = 0.3180 } ---------------------------- total value len = 21.388929 GB avg = 0.199 KB (by entry num) total key len = 2.934048 GB avg = 0.027 KB total ukey len = 2.048743 GB avg = 0.017 KB total ukey num = 0.120845783 Billion total entry num = 0.107653512 Billion write speed all = 0.517566570 MB/sec (with seq num) write speed all = 0.499240571 MB/sec (without seq num) 2024/11/23-21:58:27.279234 139957273912896 [compaction_job.cc:1948] [default] [JOB 36558] Generated table #282265: 517718 keys, 170727613 bytes, temperature: kWarm 2024/11/23-21:58:27.279301 139957273912896 EVENT_LOG_v1 {"time_micros": 1732370307279266, "cf_name": "default", "job": 36558, "event": "table_file_creation", "file_number": 282265, "file_size": 170727613, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164581136, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354360, "raw_average_key_size": 20, "raw_value_size": 526517152, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 517718, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370299, "oldest_key_time": 0, "file_creation_time": 1732370299, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282265, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:58:27.375398 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282266.sst) /dev/shm/ToplingTemp/Topling-199246-9m7gxg 2024/11/23-21:58:30.130734 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282266.sst): first pass time = 2.76's, 194.851'MB/sec 2024/11/23-21:58:34.642508 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282266.sst): old prealloc_size = 590558003, real_size = 165142256 2024/11/23-21:58:34.642837 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282266.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517725 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1641 GB avg-val =1016.98 avg-zval = 317.04 2024/11/23-21:58:34.642877 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282266.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:58:35.290386 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282266.sst): second pass time = 4.51's, 116.753'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 262.846'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3073.019'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.275'MB/sec write SST data time = 0.18's, 965.997'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.272'MB, throughput = 160349.717'MB/sec zip my value time = 4.51's, unzip length = 0.527'GB zip my value throughput = 116.753'MB/sec zip pipeline throughput = 116.753'MB/sec entries = 517725 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1016.98 avg-zval = 317.04 usrkeys = 517725 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1016.98 avg-zval = 317.04 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1641 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1367 value = 3.2077 dict = 1.00 all = 3.1527 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 21.915446 GB avg = 0.203 KB (by entry num) total key len = 2.944403 GB avg = 0.027 KB total ukey len = 2.054955 GB avg = 0.017 KB total ukey num = 0.121363508 Billion total entry num = 0.108171237 Billion write speed all = 0.528898334 MB/sec (with seq num) write speed all = 0.510487414 MB/sec (without seq num) 2024/11/23-21:58:35.475364 139957273912896 [compaction_job.cc:1948] [default] [JOB 36558] Generated table #282266: 517725 keys, 170288030 bytes, temperature: kWarm 2024/11/23-21:58:35.475435 139957273912896 EVENT_LOG_v1 {"time_micros": 1732370315475399, "cf_name": "default", "job": 36558, "event": "table_file_creation", "file_number": 282266, "file_size": 170288030, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164142112, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354500, "raw_average_key_size": 20, "raw_value_size": 526516515, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 517725, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370307, "oldest_key_time": 0, "file_creation_time": 1732370307, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282266, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:58:35.581020 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282267.sst) /dev/shm/ToplingTemp/Topling-199246-qg4Coe 2024/11/23-21:58:38.306353 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282267.sst): first pass time = 2.73's, 196.996'MB/sec 2024/11/23-21:58:42.844671 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282267.sst): old prealloc_size = 590558003, real_size = 165158672 2024/11/23-21:58:42.844993 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282267.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517711 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1642 GB avg-val =1017.01 avg-zval = 317.09 2024/11/23-21:58:42.845032 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282267.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:58:43.484674 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282267.sst): second pass time = 4.54's, 116.073'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 266.109'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3081.079'MB/sec (index lex order gen) rebuild zvType time = 0.46's, 0.279'MB/sec write SST data time = 0.18's, 970.625'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.272'MB, throughput = 135465.901'MB/sec zip my value time = 4.54's, unzip length = 0.527'GB zip my value throughput = 116.073'MB/sec zip pipeline throughput = 116.073'MB/sec entries = 517711 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.01 avg-zval = 317.09 usrkeys = 517711 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.01 avg-zval = 317.09 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1642 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1365 value = 3.2074 dict = 1.00 all = 3.1524 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 22.441962 GB avg = 0.206 KB (by entry num) total key len = 2.954757 GB avg = 0.027 KB total ukey len = 2.061168 GB avg = 0.017 KB total ukey num = 0.121881219 Billion total entry num = 0.108688948 Billion write speed all = 0.540226196 MB/sec (with seq num) write speed all = 0.521730384 MB/sec (without seq num) 2024/11/23-21:58:43.700581 139957273912896 [compaction_job.cc:1948] [default] [JOB 36558] Generated table #282267: 517711 keys, 170304227 bytes, temperature: kWarm 2024/11/23-21:58:43.700647 139957273912896 EVENT_LOG_v1 {"time_micros": 1732370323700613, "cf_name": "default", "job": 36558, "event": "table_file_creation", "file_number": 282267, "file_size": 170304227, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164158528, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354220, "raw_average_key_size": 20, "raw_value_size": 526516881, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 517711, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370315, "oldest_key_time": 0, "file_creation_time": 1732370315, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282267, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:58:43.793676 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282268.sst) /dev/shm/ToplingTemp/Topling-199246-ulXYMd 2024/11/23-21:58:46.546968 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282268.sst): first pass time = 2.75's, 194.996'MB/sec 2024/11/23-21:58:51.158807 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282268.sst): old prealloc_size = 590558003, real_size = 165159712 2024/11/23-21:58:51.159180 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282268.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517721 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1642 GB avg-val =1016.99 avg-zval = 317.08 2024/11/23-21:58:51.159225 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282268.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:58:51.805213 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282268.sst): second pass time = 4.61's, 114.208'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 263.475'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2631.314'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.277'MB/sec write SST data time = 0.18's, 951.130'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.273'MB, throughput = 160032.868'MB/sec zip my value time = 4.61's, unzip length = 0.527'GB zip my value throughput = 114.208'MB/sec zip pipeline throughput = 114.208'MB/sec entries = 517721 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1016.99 avg-zval = 317.08 usrkeys = 517721 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1016.99 avg-zval = 317.08 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1642 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1366 value = 3.2073 dict = 1.00 all = 3.1524 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 22.968479 GB avg = 0.210 KB (by entry num) total key len = 2.965112 GB avg = 0.027 KB total ukey len = 2.067380 GB avg = 0.017 KB total ukey num = 0.122398940 Billion total entry num = 0.109206669 Billion write speed all = 0.551548633 MB/sec (with seq num) write speed all = 0.532968009 MB/sec (without seq num) 2024/11/23-21:58:51.997536 139957273912896 [compaction_job.cc:1948] [default] [JOB 36558] Generated table #282268: 517721 keys, 170306543 bytes, temperature: kWarm 2024/11/23-21:58:51.997607 139957273912896 EVENT_LOG_v1 {"time_micros": 1732370331997572, "cf_name": "default", "job": 36558, "event": "table_file_creation", "file_number": 282268, "file_size": 170306543, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164159568, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354420, "raw_average_key_size": 20, "raw_value_size": 526516934, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 517721, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370323, "oldest_key_time": 0, "file_creation_time": 1732370323, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282268, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:58:52.092338 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282269.sst) /dev/shm/ToplingTemp/Topling-199246-wuwgmh 2024/11/23-21:58:52.097666 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282269.sst): first pass time = 0.01's, 1.969'MB/sec 2024/11/23-21:58:52.118747 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282269.sst): old prealloc_size = 590558003, real_size = 4072 2024/11/23-21:58:52.118783 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282269.sst): index type = UintIndex_AllOne , store type = DictZipBlobStore usrkeys = 10 prefix = 4 raw-key = 0.0000 GB zip-key = 0.0000 GB avg-key = 8.00 avg-zkey = 16.80 raw-val = 0.0000 GB zip-val = 0.0000 GB avg-val =1020.20 avg-zval = 369.60 2024/11/23-21:58:52.118960 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00/282269.sst): second pass time = 0.02's, 0.498'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.00's, 24.139'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 8.850'MB/sec (index lex order gen) rebuild zvType time = -0.00's, -0.182'MB/sec write SST data time = 0.00's, 24.970'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.572'MB/sec zip my value time = 0.02's, unzip length = 0.000'GB zip my value throughput = 0.498'MB/sec zip pipeline throughput = 0.498'MB/sec entries = 10 avg-key = 20.00 avg-zkey = 19.20 avg-val = 1020.20 avg-zval = 369.60 usrkeys = 10 avg-key = 12.00 avg-zkey = 19.20 avg-val = 1020.20 avg-zval = 369.60 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 4 UnZipSize{ index = 0.0000 GB value = 0.0000 GB dict = 0.00 MB all = 0.0000 GB } __ZipSize{ index = 0.0000 GB value = 0.0000 GB dict = 0.00 MB all = 0.0000 GB } UnZip/Zip{ index = 0.6250 value = 2.7603 dict = 1.00 all = 2.0623 } Zip/UnZip{ index = 1.6000 value = 0.3623 dict = 1.00 all = 0.4849 } ---------------------------- total value len = 22.968490 GB avg = 0.210 KB (by entry num) total key len = 2.965112 GB avg = 0.027 KB total ukey len = 2.067381 GB avg = 0.017 KB total ukey num = 0.122398950 Billion total entry num = 0.109206679 Billion write speed all = 0.551545174 MB/sec (with seq num) write speed all = 0.532964672 MB/sec (without seq num) 2024/11/23-21:58:52.124765 139957273912896 [compaction_job.cc:1948] [default] [JOB 36558] Generated table #282269: 10 keys, 5044 bytes, temperature: kWarm 2024/11/23-21:58:52.124815 139957273912896 EVENT_LOG_v1 {"time_micros": 1732370332124787, "cf_name": "default", "job": 36558, "event": "table_file_creation", "file_number": 282269, "file_size": 5044, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 3696, "index_size": 192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 200, "raw_average_key_size": 20, "raw_value_size": 10202, "raw_average_value_size": 1020, "num_data_blocks": 1, "num_entries": 10, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-UintIndex_AllOne-DictZipBlobStore", "creation_time": 1732370332, "oldest_key_time": 0, "file_creation_time": 1732370332, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282269, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:58:52.132794 139957273912896 [compaction_job.cc:703] job-36558: subcompact[0], size: 4.315892 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36558/att-00 [282238,282239,282240,282241,282242,282243,282244,282245,282246,282247,282248,282249,282250,282251,282252,282253,282254,282255,282256,282257,282258,282259,282260,282261,282262,282263,282264,282265,282266,282267,282268,282269] 2024/11/23-21:58:52.699683 139957273912896 (Original Log Time 2024/11/23-21:54:08.777816) EVENT_LOG_v1 {"time_micros": 1732370048777774, "job": 36558, "event": "compaction_started", "compaction_reason": "ManualCompaction", "files_L4": [281026, 280966, 281027, 281025, 281028, 281029, 281030, 281031, 281092, 281093, 281094, 281095, 281096, 281097, 281098, 281099, 281100, 281101, 281102, 281103, 281104, 281105, 281106, 281107, 281108, 281109, 281110, 281111, 281361, 281362, 281363, 281364, 281365, 281366, 281367, 281368, 281369, 281370, 281371, 281372, 281373, 281374, 281375, 281376, 281377, 281378, 281379, 281380, 281381, 281382, 281383, 281384, 281385, 281386, 281387, 281388, 281389, 281390, 281391, 281392], "score": -1, "input_data_size": 4298066600, "oldest_snapshot_seqno": 4284648667} 2024/11/23-21:58:52.699687 139957273912896 (Original Log Time 2024/11/23-21:58:52.697970) [compaction_job.cc:2022] [default] [JOB 36558] Compacted 60@4 files to L5 => 4315891819 bytes 2024/11/23-21:58:52.699688 139957273912896 (Original Log Time 2024/11/23-21:58:52.699602) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 0 0 32 0] max score 0.00, MB/sec: 15.2 rd, 15.2 wr, level 5, files in(60, 0) out(32 +0 blob) MB in(4099.0, 0.0 +0.0 blob) out(4116.0 +0.0 blob), read-write-amplify(2.0) write-amplify(1.0) OK, records in: 56848515, records dropped: 0 output_compression: Snappy 2024/11/23-21:58:52.699691 139957273912896 (Original Log Time 2024/11/23-21:58:52.699654) EVENT_LOG_v1 {"time_micros": 1732370332699627, "cf": "default", "job": 36558, "event": "compaction_finished", "compaction_time_micros": 283354983, "compaction_time_cpu_micros": 90754517, "output_level": 5, "num_output_files": 32, "total_output_size": 4315891819, "num_input_records": 56848515, "num_output_records": 56848515, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 2873834121, "file_range_sync_nanos": 0, "file_fsync_nanos": 22689, "file_prepare_write_nanos": 71253, "lsm_state": [0, 0, 0, 0, 0, 32, 0]} 2024/11/23-21:58:52.849963 139957273912896 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732370048,"executesMs":284081,"compactionJobId":36558,"attempt":0,"compactionInputRawBytes":15644160443,"compactionInputZipBytes":4298066600,"compactionOutputRawBytes":0,"compactionOutputZipBytes":4315891819,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-9b264cbe0e191abf9f6c42bd38168658-4ecb1818277795a7-00"}}, response = 2024/11/23-21:58:52.850093 139957273912896 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-36558/att-00: olev 5, work 284.081 s, result 7.860 ms, install 1.737 ms, input{raw 14.570 GiB zip 4.003 GiB}