2024/11/24-04:26:07.136891 139957517301312 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00: fork to child time = 0.006988 sec 2024/11/24-04:26:07.137647 139957517301312 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00 2024/11/24-04:26:07.144948 139957517301312 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00 2024/11/24-04:26:07.145856 139957517301312 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-04:26:07.146118 139957517301312 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-04:26:07.197836 139957517301312 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41511/att-00/MANIFEST-206347 2024/11/24-04:26:07.224083 139957517301312 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41511/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 291917, last_sequence is 4438518273, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 291712 2024/11/24-04:26:07.224089 139957517301312 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-04:26:07.224229 139957517301312 [version_set.cc:5721] Creating manifest 291917 2024/11/24-04:26:07.262116 139957517301312 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-04:26:07.282438 139957517301312 [compaction_job.cc:2437] [default] [JOB 41511] Compacting 1@2 + 20@3 files to L3, score 0.95, subcompactions 1 : 1 2024/11/24-04:26:07.282453 139957517301312 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 31102 Base level 2, inputs: [291856(27MB)], [291894(21MB) 291777(33MB) 291778(35MB) 291779(34MB) 291780(33MB) 291819(35MB) 291820(35MB) 291821(33MB) 291822(35MB) 291823(35MB) 291824(33MB) 291825(34MB) 291826(35MB) 291827(4425KB) 291783(34MB) 291784(33MB) 291785(35MB) 291786(30MB) 291787(34MB) 291895(33MB)] 2024/11/24-04:26:07.289783 139957517301312 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-04:26:07.289812 139957517301312 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-04:26:07.289831 139957517301312 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-04:26:07.289836 139957517301312 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-04:26:07.289847 139957517301312 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-04:26:07.754129 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291918.sst) /dev/shm/ToplingTemp/Topling-269364-XRwGod 2024/11/24-04:26:08.577392 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291918.sst): first pass time = 0.82's, 104.552'MB/sec 2024/11/24-04:26:09.623206 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291918.sst): old prealloc_size = 147639500, real_size = 20716992 2024/11/24-04:26:09.623594 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291918.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 597389 prefix = 4 raw-key = 0.0096 GB zip-key = 0.0012 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0693 GB zip-val = 0.0194 GB avg-val = 116.08 avg-zval = 32.50 2024/11/24-04:26:09.624194 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291918.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0015 GB, waited 0.000 sec, Key+Value = 0.086 GB 2024/11/24-04:26:10.179192 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291918.sst): second pass time = 1.04's, 66.895'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.56's, 41.742'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3231.989'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.316'MB/sec write SST data time = 0.08's, 281.410'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.695'MB, throughput = 19065.248'MB/sec zip my value time = 1.04's, unzip length = 0.069'GB zip my value throughput = 66.895'MB/sec zip pipeline throughput = 66.895'MB/sec entries = 597389 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.08 avg-zval = 32.50 usrkeys = 597389 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.08 avg-zval = 32.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 597389 bytes = 1941536 } seq expand size = 4181723 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0119 GB value = 0.0693 GB dict = 0.70 MB all = 0.0861 GB } __ZipSize{ index = 0.0012 GB value = 0.0194 GB dict = 0.70 MB all = 0.0232 GB } UnZip/Zip{ index = 10.3748 value = 3.5714 dict = 1.00 all = 3.7087 } Zip/UnZip{ index = 0.0964 value = 0.2800 dict = 1.00 all = 0.2696 } ---------------------------- total value len = 2251.399097 GB avg = 0.200 KB (by entry num) total key len = 306.899689 GB avg = 0.027 KB total ukey len = 237.366675 GB avg = 0.015 KB total ukey num = 15.739258552 Billion total entry num = 11.280572500 Billion write speed all = 36.412936293 MB/sec (with seq num) write speed all = 35.128461563 MB/sec (without seq num) 2024/11/24-04:26:10.214583 139957517301312 [compaction_job.cc:1948] [default] [JOB 41511] Generated table #291918: 597389 keys, 23207345 bytes, temperature: kWarm 2024/11/24-04:26:10.214707 139957517301312 EVENT_LOG_v1 {"time_micros": 1732393570214631, "cf_name": "default", "job": 41511, "event": "table_file_creation", "file_number": 291918, "file_size": 23207345, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4387251805, "largest_seqno": 4438129168, "table_properties": {"data_size": 19415856, "index_size": 1151616, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 16726892, "raw_average_key_size": 28, "raw_value_size": 69342558, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 597389, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393567, "oldest_key_time": 0, "file_creation_time": 1732393567, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291918, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:26:10.273672 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291919.sst) /dev/shm/ToplingTemp/Topling-269364-AVdNud 2024/11/24-04:26:11.431672 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291919.sst): first pass time = 1.16's, 115.911'MB/sec 2024/11/24-04:26:13.051331 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291919.sst): old prealloc_size = 147639500, real_size = 31398672 2024/11/24-04:26:13.051880 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291919.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.99 2024/11/24-04:26:13.052651 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291919.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:26:13.865889 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291919.sst): second pass time = 1.62's, 67.010'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.553'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3331.891'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 281.609'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 36085.207'MB/sec zip my value time = 1.62's, unzip length = 0.108'GB zip my value throughput = 67.010'MB/sec zip pipeline throughput = 100.366'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.99 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4458 value = 3.6886 dict = 1.00 all = 3.7834 } Zip/UnZip{ index = 0.0957 value = 0.2711 dict = 1.00 all = 0.2643 } ---------------------------- total value len = 2251.514010 GB avg = 0.200 KB (by entry num) total key len = 306.925430 GB avg = 0.027 KB total ukey len = 237.385061 GB avg = 0.015 KB total ukey num = 15.740177852 Billion total entry num = 11.281491800 Billion write speed all = 36.413027518 MB/sec (with seq num) write speed all = 35.128515514 MB/sec (without seq num) 2024/11/24-04:26:13.910842 139957517301312 [compaction_job.cc:1948] [default] [JOB 41511] Generated table #291919: 919300 keys, 35475378 bytes, temperature: kWarm 2024/11/24-04:26:13.910913 139957517301312 EVENT_LOG_v1 {"time_micros": 1732393573910877, "cf_name": "default", "job": 41511, "event": "table_file_creation", "file_number": 291919, "file_size": 35475378, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178645, "largest_seqno": 4414357721, "table_properties": {"data_size": 29408512, "index_size": 1760128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393570, "oldest_key_time": 0, "file_creation_time": 1732393570, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291919, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:26:13.945240 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291920.sst) /dev/shm/ToplingTemp/Topling-269364-wuCieg 2024/11/24-04:26:14.489761 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291920.sst): first pass time = 0.54's, 246.506'MB/sec 2024/11/24-04:26:15.829761 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291920.sst): old prealloc_size = 147639500, real_size = 33298848 2024/11/24-04:26:15.830346 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291920.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927145 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.76 avg-zval = 33.76 2024/11/24-04:26:15.831104 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291920.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:26:16.689722 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291920.sst): second pass time = 1.34's, 80.877'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.490'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3137.518'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 294.248'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33323.649'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.877'MB/sec zip pipeline throughput = 80.877'MB/sec entries = 927145 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.76 avg-zval = 33.76 usrkeys = 927145 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.76 avg-zval = 33.76 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927145 bytes = 3245024 } seq expand size = 6490015 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5117 value = 3.4584 dict = 1.00 all = 3.5889 } Zip/UnZip{ index = 0.0951 value = 0.2892 dict = 1.00 all = 0.2786 } ---------------------------- total value len = 2251.628758 GB avg = 0.200 KB (by entry num) total key len = 306.951390 GB avg = 0.027 KB total ukey len = 237.403604 GB avg = 0.015 KB total ukey num = 15.741104997 Billion total entry num = 11.282418945 Billion write speed all = 36.413566673 MB/sec (with seq num) write speed all = 35.129000732 MB/sec (without seq num) 2024/11/24-04:26:16.735188 139957517301312 [compaction_job.cc:1948] [default] [JOB 41511] Generated table #291920: 927145 keys, 37398390 bytes, temperature: kWarm 2024/11/24-04:26:16.735265 139957517301312 EVENT_LOG_v1 {"time_micros": 1732393576735230, "cf_name": "default", "job": 41511, "event": "table_file_creation", "file_number": 291920, "file_size": 37398390, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178655, "largest_seqno": 4438130568, "table_properties": {"data_size": 31302864, "index_size": 1764032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25960060, "raw_average_key_size": 28, "raw_value_size": 108257794, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927145, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393573, "oldest_key_time": 0, "file_creation_time": 1732393573, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291920, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:26:16.771257 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291921.sst) /dev/shm/ToplingTemp/Topling-269364-gCcHrg 2024/11/24-04:26:17.640843 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291921.sst): first pass time = 0.87's, 154.354'MB/sec 2024/11/24-04:26:18.953680 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291921.sst): old prealloc_size = 147639500, real_size = 31681216 2024/11/24-04:26:18.954258 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291921.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926781 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0297 GB avg-val = 116.82 avg-zval = 32.03 2024/11/24-04:26:18.955018 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291921.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:26:19.777263 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291921.sst): second pass time = 1.31's, 82.517'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.445'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3189.816'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.333'MB/sec write SST data time = 0.13's, 281.720'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30873.889'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.517'MB/sec zip pipeline throughput = 82.517'MB/sec entries = 926781 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.82 avg-zval = 32.03 usrkeys = 926781 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.82 avg-zval = 32.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926781 bytes = 3243744 } seq expand size = 6487467 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.08 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5133 value = 3.6471 dict = 1.00 all = 3.7513 } Zip/UnZip{ index = 0.0951 value = 0.2742 dict = 1.00 all = 0.2666 } ---------------------------- total value len = 2251.743513 GB avg = 0.200 KB (by entry num) total key len = 306.977340 GB avg = 0.027 KB total ukey len = 237.422140 GB avg = 0.015 KB total ukey num = 15.742031778 Billion total entry num = 11.283345726 Billion write speed all = 36.413969094 MB/sec (with seq num) write speed all = 35.129354081 MB/sec (without seq num) 2024/11/24-04:26:19.821336 139957517301312 [compaction_job.cc:1948] [default] [JOB 41511] Generated table #291921: 926781 keys, 35779458 bytes, temperature: kWarm 2024/11/24-04:26:19.821403 139957517301312 EVENT_LOG_v1 {"time_micros": 1732393579821368, "cf_name": "default", "job": 41511, "event": "table_file_creation", "file_number": 291921, "file_size": 35779458, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178675, "largest_seqno": 4438130333, "table_properties": {"data_size": 29686256, "index_size": 1763072, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25949868, "raw_average_key_size": 28, "raw_value_size": 108267994, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926781, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393576, "oldest_key_time": 0, "file_creation_time": 1732393576, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291921, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:26:19.856887 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291922.sst) /dev/shm/ToplingTemp/Topling-269364-OWTX3d 2024/11/24-04:26:20.850823 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291922.sst): first pass time = 0.99's, 135.041'MB/sec 2024/11/24-04:26:22.173431 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291922.sst): old prealloc_size = 147639500, real_size = 31485760 2024/11/24-04:26:22.174008 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291922.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.08 2024/11/24-04:26:22.174758 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291922.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:26:22.988939 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291922.sst): second pass time = 1.32's, 82.105'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.609'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3181.848'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 281.677'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 30234.911'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.105'MB/sec zip pipeline throughput = 82.105'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.92 avg-val = 118.00 avg-zval = 32.08 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.92 avg-val = 125.00 avg-zval = 32.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0356 GB } UnZip/Zip{ index = 10.4424 value = 3.6778 dict = 1.00 all = 3.7741 } Zip/UnZip{ index = 0.0958 value = 0.2719 dict = 1.00 all = 0.2650 } ---------------------------- total value len = 2251.858426 GB avg = 0.200 KB (by entry num) total key len = 307.003080 GB avg = 0.027 KB total ukey len = 237.440526 GB avg = 0.015 KB total ukey num = 15.742951078 Billion total entry num = 11.284265026 Billion write speed all = 36.414306402 MB/sec (with seq num) write speed all = 35.129645444 MB/sec (without seq num) 2024/11/24-04:26:23.035036 139957517301312 [compaction_job.cc:1948] [default] [JOB 41511] Generated table #291922: 919300 keys, 35562482 bytes, temperature: kWarm 2024/11/24-04:26:23.035105 139957517301312 EVENT_LOG_v1 {"time_micros": 1732393583035070, "cf_name": "default", "job": 41511, "event": "table_file_creation", "file_number": 291922, "file_size": 35562482, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4275581133, "largest_seqno": 4428202598, "table_properties": {"data_size": 29495024, "index_size": 1760704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393579, "oldest_key_time": 0, "file_creation_time": 1732393579, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291922, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:26:23.069632 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291923.sst) /dev/shm/ToplingTemp/Topling-269364-sNcHXc 2024/11/24-04:26:24.035378 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291923.sst): first pass time = 0.97's, 138.983'MB/sec 2024/11/24-04:26:25.401461 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291923.sst): old prealloc_size = 147639500, real_size = 33317824 2024/11/24-04:26:25.402039 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291923.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926940 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.80 avg-zval = 33.79 2024/11/24-04:26:25.402801 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291923.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:26:26.255368 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291923.sst): second pass time = 1.36's, 79.319'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.819'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3177.158'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 298.034'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33056.079'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.319'MB/sec zip pipeline throughput = 79.319'MB/sec entries = 926940 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.80 avg-zval = 33.79 usrkeys = 926940 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.80 avg-zval = 33.79 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926940 bytes = 3244304 } seq expand size = 6488580 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5128 value = 3.4564 dict = 1.00 all = 3.5872 } Zip/UnZip{ index = 0.0951 value = 0.2893 dict = 1.00 all = 0.2788 } ---------------------------- total value len = 2251.973178 GB avg = 0.200 KB (by entry num) total key len = 307.029034 GB avg = 0.027 KB total ukey len = 237.459064 GB avg = 0.015 KB total ukey num = 15.743878018 Billion total entry num = 11.285191966 Billion write speed all = 36.414616070 MB/sec (with seq num) write speed all = 35.129909301 MB/sec (without seq num) 2024/11/24-04:26:26.300031 139957517301312 [compaction_job.cc:1948] [default] [JOB 41511] Generated table #291923: 926940 keys, 37416148 bytes, temperature: kWarm 2024/11/24-04:26:26.300102 139957517301312 EVENT_LOG_v1 {"time_micros": 1732393586300065, "cf_name": "default", "job": 41511, "event": "table_file_creation", "file_number": 291923, "file_size": 37416148, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178686, "largest_seqno": 4438130674, "table_properties": {"data_size": 31322480, "index_size": 1763456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25954320, "raw_average_key_size": 28, "raw_value_size": 108263432, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926940, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393583, "oldest_key_time": 0, "file_creation_time": 1732393583, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291923, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:26:26.339009 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291924.sst) /dev/shm/ToplingTemp/Topling-269364-vn5AMc 2024/11/24-04:26:27.162908 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291924.sst): first pass time = 0.82's, 162.912'MB/sec 2024/11/24-04:26:28.485042 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291924.sst): old prealloc_size = 147639500, real_size = 32480048 2024/11/24-04:26:28.485633 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291924.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927137 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0305 GB avg-val = 116.77 avg-zval = 32.88 2024/11/24-04:26:28.486433 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291924.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:26:29.324276 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291924.sst): second pass time = 1.32's, 81.933'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.589'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3142.918'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.13's, 290.485'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32036.590'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.933'MB/sec zip pipeline throughput = 81.933'MB/sec entries = 927137 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.77 avg-zval = 32.88 usrkeys = 927137 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.77 avg-zval = 32.88 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927137 bytes = 3244992 } seq expand size = 6489959 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5127 value = 3.5513 dict = 1.00 all = 3.6692 } Zip/UnZip{ index = 0.0951 value = 0.2816 dict = 1.00 all = 0.2725 } ---------------------------- total value len = 2252.087926 GB avg = 0.200 KB (by entry num) total key len = 307.054994 GB avg = 0.027 KB total ukey len = 237.477607 GB avg = 0.015 KB total ukey num = 15.744805155 Billion total entry num = 11.286119103 Billion write speed all = 36.415028080 MB/sec (with seq num) write speed all = 35.130271872 MB/sec (without seq num) 2024/11/24-04:26:29.368891 139957517301312 [compaction_job.cc:1948] [default] [JOB 41511] Generated table #291924: 927137 keys, 36579586 bytes, temperature: kWarm 2024/11/24-04:26:29.368965 139957517301312 EVENT_LOG_v1 {"time_micros": 1732393589368926, "cf_name": "default", "job": 41511, "event": "table_file_creation", "file_number": 291924, "file_size": 36579586, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178700, "largest_seqno": 4438129931, "table_properties": {"data_size": 30484256, "index_size": 1763840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25959836, "raw_average_key_size": 28, "raw_value_size": 108257962, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927137, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393586, "oldest_key_time": 0, "file_creation_time": 1732393586, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291924, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:26:29.408008 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291925.sst) /dev/shm/ToplingTemp/Topling-269364-lf2oZf 2024/11/24-04:26:30.403455 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291925.sst): first pass time = 1.00's, 134.837'MB/sec 2024/11/24-04:26:31.698246 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291925.sst): old prealloc_size = 147639500, real_size = 31448576 2024/11/24-04:26:31.698769 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291925.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919809 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 117.92 avg-zval = 32.01 2024/11/24-04:26:31.699539 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291925.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:26:32.511407 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291925.sst): second pass time = 1.29's, 83.846'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.690'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3558.739'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.12's, 287.233'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 27400.995'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.846'MB/sec zip pipeline throughput = 83.846'MB/sec entries = 919809 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.92 avg-zval = 32.01 usrkeys = 919809 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.92 avg-zval = 32.01 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919809 bytes = 3219344 } seq expand size = 6438663 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.3777 value = 3.6835 dict = 1.00 all = 3.7779 } Zip/UnZip{ index = 0.0964 value = 0.2715 dict = 1.00 all = 0.2647 } ---------------------------- total value len = 2252.202828 GB avg = 0.200 KB (by entry num) total key len = 307.080749 GB avg = 0.027 KB total ukey len = 237.496003 GB avg = 0.015 KB total ukey num = 15.745724964 Billion total entry num = 11.287038912 Billion write speed all = 36.415378059 MB/sec (with seq num) write speed all = 35.130575412 MB/sec (without seq num) 2024/11/24-04:26:32.554157 139957517301312 [compaction_job.cc:1948] [default] [JOB 41511] Generated table #291925: 919809 keys, 35526714 bytes, temperature: kWarm 2024/11/24-04:26:32.554233 139957517301312 EVENT_LOG_v1 {"time_micros": 1732393592554191, "cf_name": "default", "job": 41511, "event": "table_file_creation", "file_number": 291925, "file_size": 35526714, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4289202047, "largest_seqno": 4438129516, "table_properties": {"data_size": 29445744, "index_size": 1772672, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25754652, "raw_average_key_size": 28, "raw_value_size": 108463206, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 919809, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393589, "oldest_key_time": 0, "file_creation_time": 1732393589, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291925, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:26:32.590318 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291926.sst) /dev/shm/ToplingTemp/Topling-269364-pQxveh 2024/11/24-04:26:33.608388 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291926.sst): first pass time = 1.02's, 131.841'MB/sec 2024/11/24-04:26:34.953562 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291926.sst): old prealloc_size = 147639500, real_size = 33089776 2024/11/24-04:26:34.954156 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291926.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926452 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 116.87 avg-zval = 33.56 2024/11/24-04:26:34.954921 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291926.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:26:35.800126 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291926.sst): second pass time = 1.34's, 80.565'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.928'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3083.950'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.12's, 299.447'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31322.700'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.565'MB/sec zip pipeline throughput = 80.565'MB/sec entries = 926452 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.87 avg-zval = 33.56 usrkeys = 926452 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.87 avg-zval = 33.56 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926452 bytes = 3242592 } seq expand size = 6485164 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5114 value = 3.4821 dict = 1.00 all = 3.6093 } Zip/UnZip{ index = 0.0951 value = 0.2872 dict = 1.00 all = 0.2771 } ---------------------------- total value len = 2252.317590 GB avg = 0.200 KB (by entry num) total key len = 307.106690 GB avg = 0.027 KB total ukey len = 237.514532 GB avg = 0.015 KB total ukey num = 15.746651416 Billion total entry num = 11.287965364 Billion write speed all = 36.415676040 MB/sec (with seq num) write speed all = 35.130828058 MB/sec (without seq num) 2024/11/24-04:26:35.844875 139957517301312 [compaction_job.cc:1948] [default] [JOB 41511] Generated table #291926: 926452 keys, 37186708 bytes, temperature: kWarm 2024/11/24-04:26:35.844948 139957517301312 EVENT_LOG_v1 {"time_micros": 1732393595844911, "cf_name": "default", "job": 41511, "event": "table_file_creation", "file_number": 291926, "file_size": 37186708, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178719, "largest_seqno": 4438129763, "table_properties": {"data_size": 31095200, "index_size": 1762752, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25940656, "raw_average_key_size": 28, "raw_value_size": 108277164, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926452, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393592, "oldest_key_time": 0, "file_creation_time": 1732393592, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291926, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:26:35.884548 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291927.sst) /dev/shm/ToplingTemp/Topling-269364-s3ELhd 2024/11/24-04:26:36.783480 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291927.sst): first pass time = 0.90's, 149.314'MB/sec 2024/11/24-04:26:38.147776 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291927.sst): old prealloc_size = 147639500, real_size = 32895216 2024/11/24-04:26:38.148324 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291927.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927367 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 116.73 avg-zval = 33.32 2024/11/24-04:26:38.149103 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291927.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:26:38.999744 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291927.sst): second pass time = 1.36's, 79.393'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.425'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3348.341'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.12's, 298.053'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33378.085'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.393'MB/sec zip pipeline throughput = 79.393'MB/sec entries = 927367 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.73 avg-zval = 33.32 usrkeys = 927367 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.73 avg-zval = 33.32 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927367 bytes = 3245792 } seq expand size = 6491569 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5130 value = 3.5034 dict = 1.00 all = 3.6279 } Zip/UnZip{ index = 0.0951 value = 0.2854 dict = 1.00 all = 0.2756 } ---------------------------- total value len = 2252.432333 GB avg = 0.200 KB (by entry num) total key len = 307.132656 GB avg = 0.027 KB total ukey len = 237.533080 GB avg = 0.015 KB total ukey num = 15.747578783 Billion total entry num = 11.288892731 Billion write speed all = 36.416020247 MB/sec (with seq num) write speed all = 35.131125202 MB/sec (without seq num) 2024/11/24-04:26:39.046804 139957517301312 [compaction_job.cc:1948] [default] [JOB 41511] Generated table #291927: 927367 keys, 36995506 bytes, temperature: kWarm 2024/11/24-04:26:39.046873 139957517301312 EVENT_LOG_v1 {"time_micros": 1732393599046838, "cf_name": "default", "job": 41511, "event": "table_file_creation", "file_number": 291927, "file_size": 36995506, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178730, "largest_seqno": 4438130012, "table_properties": {"data_size": 30898976, "index_size": 1764224, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25966276, "raw_average_key_size": 28, "raw_value_size": 108251502, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927367, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393595, "oldest_key_time": 0, "file_creation_time": 1732393595, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291927, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:26:39.086089 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291928.sst) /dev/shm/ToplingTemp/Topling-269364-15MYxg 2024/11/24-04:26:40.050604 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291928.sst): first pass time = 0.96's, 139.161'MB/sec 2024/11/24-04:26:41.363776 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291928.sst): old prealloc_size = 147639500, real_size = 31358576 2024/11/24-04:26:41.364372 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291928.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921774 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0294 GB avg-val = 117.61 avg-zval = 31.86 2024/11/24-04:26:41.365128 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291928.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:26:42.182059 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291928.sst): second pass time = 1.31's, 82.633'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.314'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3058.870'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.13's, 281.035'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 33567.512'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.633'MB/sec zip pipeline throughput = 82.633'MB/sec entries = 921774 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.61 avg-zval = 31.86 usrkeys = 921774 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.61 avg-zval = 31.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921774 bytes = 3226224 } seq expand size = 6452418 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.08 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4462 value = 3.6920 dict = 1.00 all = 3.7870 } Zip/UnZip{ index = 0.0957 value = 0.2709 dict = 1.00 all = 0.2641 } ---------------------------- total value len = 2252.547194 GB avg = 0.200 KB (by entry num) total key len = 307.158466 GB avg = 0.027 KB total ukey len = 237.551515 GB avg = 0.015 KB total ukey num = 15.748500557 Billion total entry num = 11.289814505 Billion write speed all = 36.416372829 MB/sec (with seq num) write speed all = 35.131431046 MB/sec (without seq num) 2024/11/24-04:26:42.231535 139957517301312 [compaction_job.cc:1948] [default] [JOB 41511] Generated table #291928: 921774 keys, 35442012 bytes, temperature: kWarm 2024/11/24-04:26:42.231612 139957517301312 EVENT_LOG_v1 {"time_micros": 1732393602231577, "cf_name": "default", "job": 41511, "event": "table_file_creation", "file_number": 291928, "file_size": 35442012, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4300937690, "largest_seqno": 4438129544, "table_properties": {"data_size": 29363168, "index_size": 1764800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25809672, "raw_average_key_size": 28, "raw_value_size": 108408152, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921774, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393599, "oldest_key_time": 0, "file_creation_time": 1732393599, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291928, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:26:42.266557 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291929.sst) /dev/shm/ToplingTemp/Topling-269364-vaxSQc 2024/11/24-04:26:43.280162 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291929.sst): first pass time = 1.01's, 132.421'MB/sec 2024/11/24-04:26:44.633190 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291929.sst): old prealloc_size = 147639500, real_size = 32944976 2024/11/24-04:26:44.633757 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291929.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924524 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 117.18 avg-zval = 33.48 2024/11/24-04:26:44.634534 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291929.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:26:45.475577 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291929.sst): second pass time = 1.35's, 80.141'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.968'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3323.744'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.12's, 296.925'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32314.673'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.141'MB/sec zip pipeline throughput = 80.141'MB/sec entries = 924524 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.18 avg-zval = 33.48 usrkeys = 924524 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.18 avg-zval = 33.48 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924524 bytes = 3235856 } seq expand size = 6471668 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5125 value = 3.4997 dict = 1.00 all = 3.6239 } Zip/UnZip{ index = 0.0951 value = 0.2857 dict = 1.00 all = 0.2759 } ---------------------------- total value len = 2252.661996 GB avg = 0.200 KB (by entry num) total key len = 307.184352 GB avg = 0.027 KB total ukey len = 237.570006 GB avg = 0.015 KB total ukey num = 15.749425081 Billion total entry num = 11.290739029 Billion write speed all = 36.416668043 MB/sec (with seq num) write speed all = 35.131681245 MB/sec (without seq num) 2024/11/24-04:26:45.525393 139957517301312 [compaction_job.cc:1948] [default] [JOB 41511] Generated table #291929: 924524 keys, 37036688 bytes, temperature: kWarm 2024/11/24-04:26:45.525463 139957517301312 EVENT_LOG_v1 {"time_micros": 1732393605525428, "cf_name": "default", "job": 41511, "event": "table_file_creation", "file_number": 291929, "file_size": 37036688, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680186, "largest_seqno": 4438130423, "table_properties": {"data_size": 30954752, "index_size": 1758912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25886672, "raw_average_key_size": 28, "raw_value_size": 108331164, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924524, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393602, "oldest_key_time": 0, "file_creation_time": 1732393602, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291929, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:26:45.560888 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291930.sst) /dev/shm/ToplingTemp/Topling-269364-HhNUJd 2024/11/24-04:26:46.509682 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291930.sst): first pass time = 0.95's, 141.469'MB/sec 2024/11/24-04:26:47.858690 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291930.sst): old prealloc_size = 147639500, real_size = 33214512 2024/11/24-04:26:47.859266 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291930.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927109 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 116.77 avg-zval = 33.67 2024/11/24-04:26:47.860026 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291930.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:26:48.709892 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291930.sst): second pass time = 1.35's, 80.298'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.838'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3170.186'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.12's, 301.437'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33325.952'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.298'MB/sec zip pipeline throughput = 80.298'MB/sec entries = 927109 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.77 avg-zval = 33.67 usrkeys = 927109 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.77 avg-zval = 33.67 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927109 bytes = 3244896 } seq expand size = 6489763 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5139 value = 3.4677 dict = 1.00 all = 3.5970 } Zip/UnZip{ index = 0.0951 value = 0.2884 dict = 1.00 all = 0.2780 } ---------------------------- total value len = 2252.776745 GB avg = 0.200 KB (by entry num) total key len = 307.210311 GB avg = 0.027 KB total ukey len = 237.588548 GB avg = 0.015 KB total ukey num = 15.750352190 Billion total entry num = 11.291666138 Billion write speed all = 36.416994156 MB/sec (with seq num) write speed all = 35.131960971 MB/sec (without seq num) 2024/11/24-04:26:48.756966 139957517301312 [compaction_job.cc:1948] [default] [JOB 41511] Generated table #291930: 927109 keys, 37314082 bytes, temperature: kWarm 2024/11/24-04:26:48.757043 139957517301312 EVENT_LOG_v1 {"time_micros": 1732393608757007, "cf_name": "default", "job": 41511, "event": "table_file_creation", "file_number": 291930, "file_size": 37314082, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680199, "largest_seqno": 4438129558, "table_properties": {"data_size": 31218976, "index_size": 1763584, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25959052, "raw_average_key_size": 28, "raw_value_size": 108258762, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927109, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393605, "oldest_key_time": 0, "file_creation_time": 1732393605, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291930, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:26:48.793746 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291931.sst) /dev/shm/ToplingTemp/Topling-269364-MuVTPg 2024/11/24-04:26:49.835547 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291931.sst): first pass time = 1.04's, 128.837'MB/sec 2024/11/24-04:26:51.151272 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291931.sst): old prealloc_size = 147639500, real_size = 30842256 2024/11/24-04:26:51.151789 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291931.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925463 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0288 GB avg-val = 117.03 avg-zval = 31.16 2024/11/24-04:26:51.152538 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291931.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:26:51.963754 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291931.sst): second pass time = 1.31's, 82.367'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 42.857'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3600.558'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.337'MB/sec write SST data time = 0.13's, 275.473'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34869.964'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.367'MB/sec zip pipeline throughput = 82.367'MB/sec entries = 925463 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.03 avg-zval = 31.16 usrkeys = 925463 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.03 avg-zval = 31.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925463 bytes = 3123456 } seq expand size = 6478241 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0288 GB dict = 1.08 MB all = 0.0348 GB } UnZip/Zip{ index = 10.4456 value = 3.7555 dict = 1.00 all = 3.8547 } Zip/UnZip{ index = 0.0957 value = 0.2663 dict = 1.00 all = 0.2594 } ---------------------------- total value len = 2252.891528 GB avg = 0.200 KB (by entry num) total key len = 307.236224 GB avg = 0.027 KB total ukey len = 237.607057 GB avg = 0.015 KB total ukey num = 15.751277653 Billion total entry num = 11.292591601 Billion write speed all = 36.417309947 MB/sec (with seq num) write speed all = 35.132230925 MB/sec (without seq num) 2024/11/24-04:26:52.007986 139957517301312 [compaction_job.cc:1948] [default] [JOB 41511] Generated table #291931: 925463 keys, 34819254 bytes, temperature: kWarm 2024/11/24-04:26:52.008061 139957517301312 EVENT_LOG_v1 {"time_micros": 1732393612008022, "cf_name": "default", "job": 41511, "event": "table_file_creation", "file_number": 291931, "file_size": 34819254, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4322920738, "largest_seqno": 4438129573, "table_properties": {"data_size": 28838720, "index_size": 1771968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25912964, "raw_average_key_size": 28, "raw_value_size": 108304802, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925463, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393608, "oldest_key_time": 0, "file_creation_time": 1732393608, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291931, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:26:52.047046 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291932.sst) /dev/shm/ToplingTemp/Topling-269364-0wT3Vc 2024/11/24-04:26:53.045746 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291932.sst): first pass time = 1.00's, 134.398'MB/sec 2024/11/24-04:26:54.346252 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291932.sst): old prealloc_size = 147639500, real_size = 31937440 2024/11/24-04:26:54.346816 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291932.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920616 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0300 GB avg-val = 117.79 avg-zval = 32.54 2024/11/24-04:26:54.347589 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291932.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:26:55.166389 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291932.sst): second pass time = 1.30's, 83.430'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.920'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3245.559'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.13's, 286.666'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 33232.981'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.430'MB/sec zip pipeline throughput = 83.430'MB/sec entries = 920616 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.79 avg-zval = 32.54 usrkeys = 920616 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.79 avg-zval = 32.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 920616 bytes = 3222176 } seq expand size = 6444312 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.5155 value = 3.6200 dict = 1.00 all = 3.7263 } Zip/UnZip{ index = 0.0951 value = 0.2762 dict = 1.00 all = 0.2684 } ---------------------------- total value len = 2253.006413 GB avg = 0.199 KB (by entry num) total key len = 307.262001 GB avg = 0.027 KB total ukey len = 237.625469 GB avg = 0.015 KB total ukey num = 15.752198269 Billion total entry num = 11.293512217 Billion write speed all = 36.417651765 MB/sec (with seq num) write speed all = 35.132526525 MB/sec (without seq num) 2024/11/24-04:26:55.211238 139957517301312 [compaction_job.cc:1948] [default] [JOB 41511] Generated table #291932: 920616 keys, 36019474 bytes, temperature: kWarm 2024/11/24-04:26:55.211310 139957517301312 EVENT_LOG_v1 {"time_micros": 1732393615211274, "cf_name": "default", "job": 41511, "event": "table_file_creation", "file_number": 291932, "file_size": 36019474, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680212, "largest_seqno": 4438130623, "table_properties": {"data_size": 29956112, "index_size": 1750976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25777248, "raw_average_key_size": 28, "raw_value_size": 108440596, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920616, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393612, "oldest_key_time": 0, "file_creation_time": 1732393612, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291932, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:26:55.251015 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291933.sst) /dev/shm/ToplingTemp/Topling-269364-GoAwcd 2024/11/24-04:26:56.238407 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291933.sst): first pass time = 0.99's, 135.937'MB/sec 2024/11/24-04:26:57.608167 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291933.sst): old prealloc_size = 147639500, real_size = 33607808 2024/11/24-04:26:57.608725 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291933.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927186 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 116.76 avg-zval = 34.08 2024/11/24-04:26:57.609496 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291933.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:26:58.461330 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291933.sst): second pass time = 1.37's, 79.106'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.203'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3340.967'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 301.519'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32377.146'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.106'MB/sec zip pipeline throughput = 79.106'MB/sec entries = 927186 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.76 avg-zval = 34.08 usrkeys = 927186 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.76 avg-zval = 34.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927186 bytes = 3245168 } seq expand size = 6490302 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.4450 value = 3.4258 dict = 1.00 all = 3.5591 } Zip/UnZip{ index = 0.0957 value = 0.2919 dict = 1.00 all = 0.2810 } ---------------------------- total value len = 2253.121160 GB avg = 0.199 KB (by entry num) total key len = 307.287963 GB avg = 0.027 KB total ukey len = 237.644013 GB avg = 0.015 KB total ukey num = 15.753125455 Billion total entry num = 11.294439403 Billion write speed all = 36.417946392 MB/sec (with seq num) write speed all = 35.132775877 MB/sec (without seq num) 2024/11/24-04:26:58.508518 139957517301312 [compaction_job.cc:1948] [default] [JOB 41511] Generated table #291933: 927186 keys, 37711388 bytes, temperature: kWarm 2024/11/24-04:26:58.508586 139957517301312 EVENT_LOG_v1 {"time_micros": 1732393618508551, "cf_name": "default", "job": 41511, "event": "table_file_creation", "file_number": 291933, "file_size": 37711388, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4438130075, "table_properties": {"data_size": 31600496, "index_size": 1775360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25961208, "raw_average_key_size": 28, "raw_value_size": 108256524, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927186, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393615, "oldest_key_time": 0, "file_creation_time": 1732393615, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291933, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:26:58.544561 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291934.sst) /dev/shm/ToplingTemp/Topling-269364-ApmbOc 2024/11/24-04:26:59.367234 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291934.sst): first pass time = 0.82's, 163.157'MB/sec 2024/11/24-04:27:00.646060 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291934.sst): old prealloc_size = 147639500, real_size = 28700320 2024/11/24-04:27:00.646666 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291934.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 940399 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1079 GB zip-val = 0.0267 GB avg-val = 114.72 avg-zval = 28.35 2024/11/24-04:27:00.647462 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291934.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:27:01.497664 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291934.sst): second pass time = 1.28's, 84.418'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 38.561'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3093.357'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.13's, 260.233'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 31999.941'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.418'MB/sec zip pipeline throughput = 84.418'MB/sec entries = 940399 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.72 avg-zval = 28.35 usrkeys = 940399 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.72 avg-zval = 28.35 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 940399 bytes = 3291408 } seq expand size = 6582793 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0188 GB value = 0.1079 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0267 GB dict = 1.08 MB all = 0.0328 GB } UnZip/Zip{ index = 10.4441 value = 4.0461 dict = 1.00 all = 4.0874 } Zip/UnZip{ index = 0.0957 value = 0.2472 dict = 1.00 all = 0.2447 } ---------------------------- total value len = 2253.235629 GB avg = 0.199 KB (by entry num) total key len = 307.314294 GB avg = 0.027 KB total ukey len = 237.662821 GB avg = 0.015 KB total ukey num = 15.754065854 Billion total entry num = 11.295379802 Billion write speed all = 36.418376257 MB/sec (with seq num) write speed all = 35.133154242 MB/sec (without seq num) 2024/11/24-04:27:01.540090 139957517301312 [compaction_job.cc:1948] [default] [JOB 41511] Generated table #291934: 940399 keys, 32837262 bytes, temperature: kWarm 2024/11/24-04:27:01.540160 139957517301312 EVENT_LOG_v1 {"time_micros": 1732393621540125, "cf_name": "default", "job": 41511, "event": "table_file_creation", "file_number": 291934, "file_size": 32837262, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680463, "largest_seqno": 4438130151, "table_properties": {"data_size": 26664208, "index_size": 1800832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26331172, "raw_average_key_size": 28, "raw_value_size": 107886642, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 940399, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393618, "oldest_key_time": 0, "file_creation_time": 1732393618, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291934, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:27:01.577215 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291935.sst) /dev/shm/ToplingTemp/Topling-269364-1HPj0e 2024/11/24-04:27:02.587279 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291935.sst): first pass time = 1.01's, 132.888'MB/sec 2024/11/24-04:27:03.875952 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291935.sst): old prealloc_size = 147639500, real_size = 29483968 2024/11/24-04:27:03.876512 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291935.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 942356 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1078 GB zip-val = 0.0275 GB avg-val = 114.43 avg-zval = 29.13 2024/11/24-04:27:03.877304 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291935.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:27:04.719302 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291935.sst): second pass time = 1.29's, 83.730'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 39.875'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3458.572'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.329'MB/sec write SST data time = 0.13's, 267.426'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.075'MB, throughput = 32580.786'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.730'MB/sec zip pipeline throughput = 83.730'MB/sec entries = 942356 avg-key = 28.00 avg-zkey = 1.90 avg-val = 114.43 avg-zval = 29.13 usrkeys = 942356 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.43 avg-zval = 29.13 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 942356 bytes = 3298256 } seq expand size = 6596492 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0188 GB value = 0.1078 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0275 GB dict = 1.08 MB all = 0.0336 GB } UnZip/Zip{ index = 10.5129 value = 3.9275 dict = 1.00 all = 3.9913 } Zip/UnZip{ index = 0.0951 value = 0.2546 dict = 1.00 all = 0.2505 } ---------------------------- total value len = 2253.350057 GB avg = 0.199 KB (by entry num) total key len = 307.340680 GB avg = 0.027 KB total ukey len = 237.681668 GB avg = 0.015 KB total ukey num = 15.755008210 Billion total entry num = 11.296322158 Billion write speed all = 36.418710301 MB/sec (with seq num) write speed all = 35.133439954 MB/sec (without seq num) 2024/11/24-04:27:04.761569 139957517301312 [compaction_job.cc:1948] [default] [JOB 41511] Generated table #291935: 942356 keys, 33627922 bytes, temperature: kWarm 2024/11/24-04:27:04.761640 139957517301312 EVENT_LOG_v1 {"time_micros": 1732393624761604, "cf_name": "default", "job": 41511, "event": "table_file_creation", "file_number": 291935, "file_size": 33627922, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4438130365, "table_properties": {"data_size": 27455408, "index_size": 1792768, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26385968, "raw_average_key_size": 28, "raw_value_size": 107831892, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 942356, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393621, "oldest_key_time": 0, "file_creation_time": 1732393621, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291935, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:27:04.812773 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291936.sst) /dev/shm/ToplingTemp/Topling-269364-lRjiRd 2024/11/24-04:27:05.807373 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291936.sst): first pass time = 0.99's, 134.952'MB/sec 2024/11/24-04:27:07.096838 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291936.sst): old prealloc_size = 147639500, real_size = 31430384 2024/11/24-04:27:07.097414 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291936.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 32.02 2024/11/24-04:27:07.098172 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291936.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:27:07.914002 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291936.sst): second pass time = 1.29's, 84.203'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.454'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3170.800'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.13's, 278.716'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 34627.242'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.203'MB/sec zip pipeline throughput = 84.203'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.02 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.02 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4443 value = 3.6847 dict = 1.00 all = 3.7800 } Zip/UnZip{ index = 0.0957 value = 0.2714 dict = 1.00 all = 0.2645 } ---------------------------- total value len = 2253.464970 GB avg = 0.199 KB (by entry num) total key len = 307.366420 GB avg = 0.027 KB total ukey len = 237.700054 GB avg = 0.015 KB total ukey num = 15.755927510 Billion total entry num = 11.297241458 Billion write speed all = 36.419055980 MB/sec (with seq num) write speed all = 35.133739436 MB/sec (without seq num) 2024/11/24-04:27:07.957326 139957517301312 [compaction_job.cc:1948] [default] [JOB 41511] Generated table #291936: 919300 keys, 35507122 bytes, temperature: kWarm 2024/11/24-04:27:07.957394 139957517301312 EVENT_LOG_v1 {"time_micros": 1732393627957359, "cf_name": "default", "job": 41511, "event": "table_file_creation", "file_number": 291936, "file_size": 35507122, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4279898035, "largest_seqno": 4425254965, "table_properties": {"data_size": 29439968, "index_size": 1760384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393624, "oldest_key_time": 0, "file_creation_time": 1732393624, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291936, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:27:08.006767 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291937.sst) /dev/shm/ToplingTemp/Topling-269364-ZkpZoh 2024/11/24-04:27:08.138244 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291937.sst): first pass time = 0.13's, 236.086'MB/sec 2024/11/24-04:27:08.435419 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291937.sst): old prealloc_size = 147639500, real_size = 8370608 2024/11/24-04:27:08.435603 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291937.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 216324 prefix = 4 raw-key = 0.0035 GB zip-key = 0.0004 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0250 GB zip-val = 0.0079 GB avg-val = 115.44 avg-zval = 36.52 2024/11/24-04:27:08.435818 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291937.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.031 GB 2024/11/24-04:27:08.631490 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00/291937.sst): second pass time = 0.30's, 84.231'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.20's, 47.472'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2547.229'MB/sec (index lex order gen) rebuild zvType time = 0.17's, 0.327'MB/sec write SST data time = 0.03's, 306.608'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.258'MB, throughput = 8806.665'MB/sec zip my value time = 0.30's, unzip length = 0.025'GB zip my value throughput = 84.231'MB/sec zip pipeline throughput = 84.231'MB/sec entries = 216324 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.44 avg-zval = 36.52 usrkeys = 216324 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.44 avg-zval = 36.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 216324 bytes = 730112 } seq expand size = 1514268 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0043 GB value = 0.0250 GB dict = 0.26 MB all = 0.0310 GB } __ZipSize{ index = 0.0004 GB value = 0.0079 GB dict = 0.26 MB all = 0.0093 GB } UnZip/Zip{ index = 10.4098 value = 3.1609 dict = 1.00 all = 3.3341 } Zip/UnZip{ index = 0.0961 value = 0.3164 dict = 1.00 all = 0.2999 } ---------------------------- total value len = 2253.491458 GB avg = 0.199 KB (by entry num) total key len = 307.372477 GB avg = 0.027 KB total ukey len = 237.704381 GB avg = 0.015 KB total ukey num = 15.756143834 Billion total entry num = 11.297457782 Billion write speed all = 36.419147192 MB/sec (with seq num) write speed all = 35.133819152 MB/sec (without seq num) 2024/11/24-04:27:08.648023 139957517301312 [compaction_job.cc:1948] [default] [JOB 41511] Generated table #291937: 216324 keys, 9306985 bytes, temperature: kWarm 2024/11/24-04:27:08.648081 139957517301312 EVENT_LOG_v1 {"time_micros": 1732393628648053, "cf_name": "default", "job": 41511, "event": "table_file_creation", "file_number": 291937, "file_size": 9306985, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4323261151, "largest_seqno": 4438130165, "table_properties": {"data_size": 7900704, "index_size": 415616, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 6057072, "raw_average_key_size": 28, "raw_value_size": 24973400, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 216324, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393627, "oldest_key_time": 0, "file_creation_time": 1732393627, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291937, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:27:08.661407 139957517301312 [compaction_job.cc:703] job-41511: subcompact[0], size: 0.680750 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41511/att-00 [291918,291919,291920,291921,291922,291923,291924,291925,291926,291927,291928,291929,291930,291931,291932,291933,291934,291935,291936,291937] 2024/11/24-04:27:08.778800 139957517301312 (Original Log Time 2024/11/24-04:26:07.282503) EVENT_LOG_v1 {"time_micros": 1732393567282465, "job": 41511, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [291856], "files_L3": [291894, 291777, 291778, 291779, 291780, 291819, 291820, 291821, 291822, 291823, 291824, 291825, 291826, 291827, 291783, 291784, 291785, 291786, 291787, 291895], "score": 0.949795, "input_data_size": 705245206, "oldest_snapshot_seqno": 4438517992} 2024/11/24-04:27:08.778810 139957517301312 (Original Log Time 2024/11/24-04:27:08.777193) [compaction_job.cc:2022] [default] [JOB 41511] Compacted 1@2 + 20@3 files to L3 => 680749904 bytes 2024/11/24-04:27:08.778811 139957517301312 (Original Log Time 2024/11/24-04:27:08.778726) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 20 0 0 0] max score 0.03, MB/sec: 11.5 rd, 11.1 wr, level 3, files in(1, 20) out(20 +0 blob) MB in(27.1, 645.5 +0.0 blob) out(649.2 +0.0 blob), read-write-amplify(48.8) write-amplify(24.0) OK, records in: 17586087, records dropped: 103416 output_compression: Snappy 2024/11/24-04:27:08.778815 139957517301312 (Original Log Time 2024/11/24-04:27:08.778774) EVENT_LOG_v1 {"time_micros": 1732393628778750, "cf": "default", "job": 41511, "event": "compaction_finished", "compaction_time_micros": 61378910, "compaction_time_cpu_micros": 26196482, "output_level": 3, "num_output_files": 20, "total_output_size": 680749904, "num_input_records": 17586087, "num_output_records": 17482671, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 471569882, "file_range_sync_nanos": 0, "file_fsync_nanos": 13056, "file_prepare_write_nanos": 51987, "lsm_state": [0, 0, 0, 20, 0, 0, 0]} 2024/11/24-04:27:08.910464 139957517301312 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732393567,"executesMs":61632,"compactionJobId":41511,"attempt":0,"compactionInputRawBytes":2547705606,"compactionInputZipBytes":705245206,"compactionOutputRawBytes":0,"compactionOutputZipBytes":680749904,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-b6763664011140875b9119325b280561-adf46b6a1fd0e76d-00"}}, response = 2024/11/24-04:27:08.910540 139957517301312 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41511/att-00: olev 3, work 61.632 s, result 7.786 ms, install 1.637 ms, input{raw 2.373 GiB zip 672.574 MiB}