2024/11/24-03:36:21.267053 139957206771264 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00: fork to child time = 0.007616 sec 2024/11/24-03:36:21.267789 139957206771264 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00 2024/11/24-03:36:21.273608 139957206771264 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00 2024/11/24-03:36:21.274492 139957206771264 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-03:36:21.274744 139957206771264 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-03:36:21.328276 139957206771264 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-40913/att-00/MANIFEST-206347 2024/11/24-03:36:21.355762 139957206771264 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-40913/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 290579, last_sequence is 4420697399, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 290429 2024/11/24-03:36:21.355768 139957206771264 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-03:36:21.355901 139957206771264 [version_set.cc:5721] Creating manifest 290579 2024/11/24-03:36:21.391184 139957206771264 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-03:36:21.414456 139957206771264 [compaction_job.cc:2437] [default] [JOB 40913] Compacting 1@2 + 23@3 files to L3, score 0.97, subcompactions 1 : 1 2024/11/24-03:36:21.414470 139957206771264 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 30642 Base level 2, inputs: [290516(27MB)], [290576(33MB) 290577(991KB) 290534(362KB) 290477(33MB) 290478(35MB) 290479(35MB) 290480(34MB) 290481(33MB) 290482(35MB) 290483(35MB) 290484(35MB) 290485(33MB) 290486(33MB) 290487(30MB) 290488(35MB) 290489(5124KB) 290453(31MB) 290454(35MB) 290455(32MB) 290456(34MB) 290457(32MB) 290458(33MB) 290549(34MB)] 2024/11/24-03:36:21.421006 139957206771264 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-03:36:21.421034 139957206771264 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-03:36:21.421053 139957206771264 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-03:36:21.421058 139957206771264 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-03:36:21.421069 139957206771264 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-03:36:22.057944 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290580.sst) /dev/shm/ToplingTemp/Topling-260105-lvsWD9 2024/11/24-03:36:22.597686 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290580.sst): first pass time = 0.54's, 248.698'MB/sec 2024/11/24-03:36:23.930279 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290580.sst): old prealloc_size = 147639500, real_size = 31215936 2024/11/24-03:36:23.930818 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290580.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926599 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0292 GB avg-val = 116.85 avg-zval = 31.52 2024/11/24-03:36:23.931616 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290580.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:36:24.746136 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290580.sst): second pass time = 1.33's, 81.304'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.286'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3448.884'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 281.831'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34955.998'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.304'MB/sec zip pipeline throughput = 81.304'MB/sec entries = 926599 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.85 avg-zval = 31.52 usrkeys = 926599 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.85 avg-zval = 31.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926599 bytes = 3243104 } seq expand size = 6486193 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4430 value = 3.7068 dict = 1.00 all = 3.8007 } Zip/UnZip{ index = 0.0958 value = 0.2698 dict = 1.00 all = 0.2631 } ---------------------------- total value len = 2251.440332 GB avg = 0.200 KB (by entry num) total key len = 306.908907 GB avg = 0.027 KB total ukey len = 237.373259 GB avg = 0.015 KB total ukey num = 15.739587762 Billion total entry num = 11.280901710 Billion write speed all = 38.029626412 MB/sec (with seq num) write speed all = 36.688109872 MB/sec (without seq num) 2024/11/24-03:36:24.789790 139957206771264 [compaction_job.cc:1948] [default] [JOB 40913] Generated table #290580: 926599 keys, 35313652 bytes, temperature: kWarm 2024/11/24-03:36:24.789893 139957206771264 EVENT_LOG_v1 {"time_micros": 1732390584789830, "cf_name": "default", "job": 40913, "event": "table_file_creation", "file_number": 290580, "file_size": 35313652, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178686, "largest_seqno": 4420282464, "table_properties": {"data_size": 29209520, "index_size": 1774592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25944772, "raw_average_key_size": 28, "raw_value_size": 108273050, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926599, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390582, "oldest_key_time": 0, "file_creation_time": 1732390582, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290580, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:36:24.828397 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290581.sst) /dev/shm/ToplingTemp/Topling-260105-JlDvj6 2024/11/24-03:36:25.773899 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290581.sst): first pass time = 0.95's, 141.960'MB/sec 2024/11/24-03:36:27.084363 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290581.sst): old prealloc_size = 147639500, real_size = 31473296 2024/11/24-03:36:27.084931 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290581.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.07 2024/11/24-03:36:27.085720 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290581.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:36:27.900029 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290581.sst): second pass time = 1.31's, 82.828'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.586'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3250.224'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.13's, 282.700'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 34664.856'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.828'MB/sec zip pipeline throughput = 82.828'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.07 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.07 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0356 GB } UnZip/Zip{ index = 10.4439 value = 3.6793 dict = 1.00 all = 3.7755 } Zip/UnZip{ index = 0.0957 value = 0.2718 dict = 1.00 all = 0.2649 } ---------------------------- total value len = 2251.555245 GB avg = 0.200 KB (by entry num) total key len = 306.934648 GB avg = 0.027 KB total ukey len = 237.391645 GB avg = 0.015 KB total ukey num = 15.740507062 Billion total entry num = 11.281821010 Billion write speed all = 38.029934275 MB/sec (with seq num) write speed all = 36.688371308 MB/sec (without seq num) 2024/11/24-03:36:27.943218 139957206771264 [compaction_job.cc:1948] [default] [JOB 40913] Generated table #290581: 919300 keys, 35550002 bytes, temperature: kWarm 2024/11/24-03:36:27.943289 139957206771264 EVENT_LOG_v1 {"time_micros": 1732390587943252, "cf_name": "default", "job": 40913, "event": "table_file_creation", "file_number": 290581, "file_size": 35550002, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4266841817, "largest_seqno": 4420003400, "table_properties": {"data_size": 29482816, "index_size": 1760448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390584, "oldest_key_time": 0, "file_creation_time": 1732390584, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290581, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:36:27.981043 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290582.sst) /dev/shm/ToplingTemp/Topling-260105-x42mV7 2024/11/24-03:36:28.871549 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290582.sst): first pass time = 0.89's, 150.729'MB/sec 2024/11/24-03:36:30.210027 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290582.sst): old prealloc_size = 147639500, real_size = 32996144 2024/11/24-03:36:30.210569 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290582.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926753 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 116.83 avg-zval = 33.45 2024/11/24-03:36:30.211328 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290582.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:36:31.059503 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290582.sst): second pass time = 1.34's, 80.966'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.668'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3397.536'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 295.821'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33996.664'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.966'MB/sec zip pipeline throughput = 80.966'MB/sec entries = 926753 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.83 avg-zval = 33.45 usrkeys = 926753 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.83 avg-zval = 33.45 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926753 bytes = 3243648 } seq expand size = 6487271 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5137 value = 3.4924 dict = 1.00 all = 3.6183 } Zip/UnZip{ index = 0.0951 value = 0.2863 dict = 1.00 all = 0.2764 } ---------------------------- total value len = 2251.670001 GB avg = 0.200 KB (by entry num) total key len = 306.960597 GB avg = 0.027 KB total ukey len = 237.410180 GB avg = 0.015 KB total ukey num = 15.741433815 Billion total entry num = 11.282747763 Billion write speed all = 38.030239720 MB/sec (with seq num) write speed all = 36.688629556 MB/sec (without seq num) 2024/11/24-03:36:31.106100 139957206771264 [compaction_job.cc:1948] [default] [JOB 40913] Generated table #290582: 926753 keys, 37094042 bytes, temperature: kWarm 2024/11/24-03:36:31.106164 139957206771264 EVENT_LOG_v1 {"time_micros": 1732390591106130, "cf_name": "default", "job": 40913, "event": "table_file_creation", "file_number": 290582, "file_size": 37094042, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178700, "largest_seqno": 4420282575, "table_properties": {"data_size": 31001312, "index_size": 1762944, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25949084, "raw_average_key_size": 28, "raw_value_size": 108268766, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926753, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390587, "oldest_key_time": 0, "file_creation_time": 1732390587, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290582, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:36:31.141382 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290583.sst) /dev/shm/ToplingTemp/Topling-260105-WZz6N8 2024/11/24-03:36:32.174259 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290583.sst): first pass time = 1.03's, 129.949'MB/sec 2024/11/24-03:36:33.519273 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290583.sst): old prealloc_size = 147639500, real_size = 33380400 2024/11/24-03:36:33.519810 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290583.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926623 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.85 avg-zval = 33.86 2024/11/24-03:36:33.520632 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290583.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:36:34.385055 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290583.sst): second pass time = 1.34's, 80.553'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 43.291'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3455.997'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.314'MB/sec write SST data time = 0.13's, 294.774'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32788.311'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.553'MB/sec zip pipeline throughput = 80.553'MB/sec entries = 926623 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.85 avg-zval = 33.86 usrkeys = 926623 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.85 avg-zval = 33.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926623 bytes = 3243200 } seq expand size = 6486361 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4432 value = 3.4510 dict = 1.00 all = 3.5812 } Zip/UnZip{ index = 0.0958 value = 0.2898 dict = 1.00 all = 0.2792 } ---------------------------- total value len = 2251.784760 GB avg = 0.200 KB (by entry num) total key len = 306.986542 GB avg = 0.027 KB total ukey len = 237.428713 GB avg = 0.015 KB total ukey num = 15.742360438 Billion total entry num = 11.283674386 Billion write speed all = 38.030451273 MB/sec (with seq num) write speed all = 36.688797242 MB/sec (without seq num) 2024/11/24-03:36:34.429575 139957206771264 [compaction_job.cc:1948] [default] [JOB 40913] Generated table #290583: 926623 keys, 37478346 bytes, temperature: kWarm 2024/11/24-03:36:34.429645 139957206771264 EVENT_LOG_v1 {"time_micros": 1732390594429609, "cf_name": "default", "job": 40913, "event": "table_file_creation", "file_number": 290583, "file_size": 37478346, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178719, "largest_seqno": 4420281958, "table_properties": {"data_size": 31373984, "index_size": 1774592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25945444, "raw_average_key_size": 28, "raw_value_size": 108272366, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926623, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390591, "oldest_key_time": 0, "file_creation_time": 1732390591, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290583, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:36:34.466698 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290584.sst) /dev/shm/ToplingTemp/Topling-260105-y68xi5 2024/11/24-03:36:35.473692 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290584.sst): first pass time = 1.01's, 133.290'MB/sec 2024/11/24-03:36:36.756187 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290584.sst): old prealloc_size = 147639500, real_size = 31390416 2024/11/24-03:36:36.756707 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290584.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926984 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0294 GB avg-val = 116.79 avg-zval = 31.71 2024/11/24-03:36:36.757483 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290584.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:36:37.579395 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290584.sst): second pass time = 1.28's, 84.490'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.112'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3503.191'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.333'MB/sec write SST data time = 0.13's, 280.158'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31517.696'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.490'MB/sec zip pipeline throughput = 84.490'MB/sec entries = 926984 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.79 avg-zval = 31.71 usrkeys = 926984 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.79 avg-zval = 31.71 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926984 bytes = 3244464 } seq expand size = 6488888 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.5114 value = 3.6831 dict = 1.00 all = 3.7819 } Zip/UnZip{ index = 0.0951 value = 0.2715 dict = 1.00 all = 0.2644 } ---------------------------- total value len = 2251.899511 GB avg = 0.200 KB (by entry num) total key len = 307.012498 GB avg = 0.027 KB total ukey len = 237.447253 GB avg = 0.015 KB total ukey num = 15.743287422 Billion total entry num = 11.284601370 Billion write speed all = 38.030736974 MB/sec (with seq num) write speed all = 36.689036422 MB/sec (without seq num) 2024/11/24-03:36:37.622035 139957206771264 [compaction_job.cc:1948] [default] [JOB 40913] Generated table #290584: 926984 keys, 35489118 bytes, temperature: kWarm 2024/11/24-03:36:37.622113 139957206771264 EVENT_LOG_v1 {"time_micros": 1732390597622068, "cf_name": "default", "job": 40913, "event": "table_file_creation", "file_number": 290584, "file_size": 35489118, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178730, "largest_seqno": 4420282699, "table_properties": {"data_size": 29394688, "index_size": 1763776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25955552, "raw_average_key_size": 28, "raw_value_size": 108262248, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926984, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390594, "oldest_key_time": 0, "file_creation_time": 1732390594, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290584, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:36:37.656514 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290585.sst) /dev/shm/ToplingTemp/Topling-260105-QJNjG7 2024/11/24-03:36:38.630227 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290585.sst): first pass time = 0.97's, 137.847'MB/sec 2024/11/24-03:36:39.913189 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290585.sst): old prealloc_size = 147639500, real_size = 31521024 2024/11/24-03:36:39.913761 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290585.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919618 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 117.95 avg-zval = 32.10 2024/11/24-03:36:39.914541 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290585.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:36:40.731633 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290585.sst): second pass time = 1.28's, 84.604'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.497'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3230.506'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.12's, 286.154'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 33459.249'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.604'MB/sec zip pipeline throughput = 84.604'MB/sec entries = 919618 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.95 avg-zval = 32.10 usrkeys = 919618 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.95 avg-zval = 32.10 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919618 bytes = 3218672 } seq expand size = 6437326 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0356 GB } UnZip/Zip{ index = 10.3796 value = 3.6745 dict = 1.00 all = 3.7703 } Zip/UnZip{ index = 0.0963 value = 0.2721 dict = 1.00 all = 0.2652 } ---------------------------- total value len = 2252.014417 GB avg = 0.200 KB (by entry num) total key len = 307.038247 GB avg = 0.027 KB total ukey len = 237.465645 GB avg = 0.015 KB total ukey num = 15.744207040 Billion total entry num = 11.285520988 Billion write speed all = 38.031045692 MB/sec (with seq num) write speed all = 36.689298661 MB/sec (without seq num) 2024/11/24-03:36:40.775549 139957206771264 [compaction_job.cc:1948] [default] [JOB 40913] Generated table #290585: 919618 keys, 35598858 bytes, temperature: kWarm 2024/11/24-03:36:40.775625 139957206771264 EVENT_LOG_v1 {"time_micros": 1732390600775586, "cf_name": "default", "job": 40913, "event": "table_file_creation", "file_number": 290585, "file_size": 35598858, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4269559142, "largest_seqno": 4420277796, "table_properties": {"data_size": 29518960, "index_size": 1771968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25749304, "raw_average_key_size": 28, "raw_value_size": 108468504, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 919618, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390597, "oldest_key_time": 0, "file_creation_time": 1732390597, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290585, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:36:40.811522 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290586.sst) /dev/shm/ToplingTemp/Topling-260105-qDUxf5 2024/11/24-03:36:41.835055 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290586.sst): first pass time = 1.02's, 131.137'MB/sec 2024/11/24-03:36:43.167290 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290586.sst): old prealloc_size = 147639500, real_size = 33041984 2024/11/24-03:36:43.167851 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290586.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926344 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 116.89 avg-zval = 33.52 2024/11/24-03:36:43.168659 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290586.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:36:44.018631 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290586.sst): second pass time = 1.33's, 81.355'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.625'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3267.155'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 297.103'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32937.818'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.355'MB/sec zip pipeline throughput = 81.355'MB/sec entries = 926344 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.89 avg-zval = 33.52 usrkeys = 926344 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.89 avg-zval = 33.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926344 bytes = 3242224 } seq expand size = 6484408 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5117 value = 3.4875 dict = 1.00 all = 3.6140 } Zip/UnZip{ index = 0.0951 value = 0.2867 dict = 1.00 all = 0.2767 } ---------------------------- total value len = 2252.129181 GB avg = 0.200 KB (by entry num) total key len = 307.064185 GB avg = 0.027 KB total ukey len = 237.484172 GB avg = 0.015 KB total ukey num = 15.745133384 Billion total entry num = 11.286447332 Billion write speed all = 38.031278920 MB/sec (with seq num) write speed all = 36.689487300 MB/sec (without seq num) 2024/11/24-03:36:44.065448 139957206771264 [compaction_job.cc:1948] [default] [JOB 40913] Generated table #290586: 926344 keys, 37138512 bytes, temperature: kWarm 2024/11/24-03:36:44.065520 139957206771264 EVENT_LOG_v1 {"time_micros": 1732390604065483, "cf_name": "default", "job": 40913, "event": "table_file_creation", "file_number": 290586, "file_size": 37138512, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680186, "largest_seqno": 4420282918, "table_properties": {"data_size": 31047728, "index_size": 1762496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25937632, "raw_average_key_size": 28, "raw_value_size": 108280208, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926344, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390600, "oldest_key_time": 0, "file_creation_time": 1732390600, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290586, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:36:44.101700 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290587.sst) /dev/shm/ToplingTemp/Topling-260105-HYFde8 2024/11/24-03:36:45.143365 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290587.sst): first pass time = 1.04's, 128.855'MB/sec 2024/11/24-03:36:46.527730 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290587.sst): old prealloc_size = 147639500, real_size = 33450960 2024/11/24-03:36:46.528303 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290587.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926758 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 116.83 avg-zval = 33.94 2024/11/24-03:36:46.529095 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290587.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:36:47.384942 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290587.sst): second pass time = 1.38's, 78.253'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.805'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3201.459'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 298.351'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 27938.919'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.253'MB/sec zip pipeline throughput = 78.253'MB/sec entries = 926758 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.83 avg-zval = 33.94 usrkeys = 926758 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.83 avg-zval = 33.94 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926758 bytes = 3243664 } seq expand size = 6487306 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5134 value = 3.4419 dict = 1.00 all = 3.5745 } Zip/UnZip{ index = 0.0951 value = 0.2905 dict = 1.00 all = 0.2798 } ---------------------------- total value len = 2252.243937 GB avg = 0.200 KB (by entry num) total key len = 307.090134 GB avg = 0.027 KB total ukey len = 237.502707 GB avg = 0.015 KB total ukey num = 15.746060142 Billion total entry num = 11.287374090 Billion write speed all = 38.031467343 MB/sec (with seq num) write speed all = 36.689632672 MB/sec (without seq num) 2024/11/24-03:36:47.430463 139957206771264 [compaction_job.cc:1948] [default] [JOB 40913] Generated table #290587: 926758 keys, 37549150 bytes, temperature: kWarm 2024/11/24-03:36:47.430539 139957206771264 EVENT_LOG_v1 {"time_micros": 1732390607430499, "cf_name": "default", "job": 40913, "event": "table_file_creation", "file_number": 290587, "file_size": 37549150, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680199, "largest_seqno": 4420282367, "table_properties": {"data_size": 31456064, "index_size": 1763008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25949224, "raw_average_key_size": 28, "raw_value_size": 108268540, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926758, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390604, "oldest_key_time": 0, "file_creation_time": 1732390604, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290587, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:36:47.464923 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290588.sst) /dev/shm/ToplingTemp/Topling-260105-qwMQQ6 2024/11/24-03:36:48.410413 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290588.sst): first pass time = 0.95's, 141.962'MB/sec 2024/11/24-03:36:49.734507 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290588.sst): old prealloc_size = 147639500, real_size = 32280496 2024/11/24-03:36:49.735119 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290588.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926463 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0303 GB avg-val = 116.87 avg-zval = 32.69 2024/11/24-03:36:49.735870 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290588.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:36:50.564375 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290588.sst): second pass time = 1.32's, 81.827'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.837'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2998.362'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.328'MB/sec write SST data time = 0.12's, 294.287'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 26420.561'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.827'MB/sec zip pipeline throughput = 81.827'MB/sec entries = 926463 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.87 avg-zval = 32.69 usrkeys = 926463 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.87 avg-zval = 32.69 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926463 bytes = 3242640 } seq expand size = 6485241 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5138 value = 3.5751 dict = 1.00 all = 3.6896 } Zip/UnZip{ index = 0.0951 value = 0.2797 dict = 1.00 all = 0.2710 } ---------------------------- total value len = 2252.358699 GB avg = 0.200 KB (by entry num) total key len = 307.116075 GB avg = 0.027 KB total ukey len = 237.521236 GB avg = 0.015 KB total ukey num = 15.746986605 Billion total entry num = 11.288300553 Billion write speed all = 38.031761323 MB/sec (with seq num) write speed all = 36.689879913 MB/sec (without seq num) 2024/11/24-03:36:50.609470 139957206771264 [compaction_job.cc:1948] [default] [JOB 40913] Generated table #290588: 926463 keys, 36377776 bytes, temperature: kWarm 2024/11/24-03:36:50.609542 139957206771264 EVENT_LOG_v1 {"time_micros": 1732390610609505, "cf_name": "default", "job": 40913, "event": "table_file_creation", "file_number": 290588, "file_size": 36377776, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680212, "largest_seqno": 4420281012, "table_properties": {"data_size": 30286304, "index_size": 1762368, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25940964, "raw_average_key_size": 28, "raw_value_size": 108276838, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926463, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390607, "oldest_key_time": 0, "file_creation_time": 1732390607, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290588, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:36:50.644417 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290589.sst) /dev/shm/ToplingTemp/Topling-260105-B6pPn8 2024/11/24-03:36:51.633643 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290589.sst): first pass time = 0.99's, 135.686'MB/sec 2024/11/24-03:36:52.950606 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290589.sst): old prealloc_size = 147639500, real_size = 31143776 2024/11/24-03:36:52.951192 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290589.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922720 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0291 GB avg-val = 117.46 avg-zval = 31.59 2024/11/24-03:36:52.951945 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290589.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:36:53.762531 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290589.sst): second pass time = 1.32's, 82.377'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.250'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3143.354'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 280.323'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33691.791'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.377'MB/sec zip pipeline throughput = 82.377'MB/sec entries = 922720 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.46 avg-zval = 31.59 usrkeys = 922720 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.46 avg-zval = 31.59 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922720 bytes = 3114192 } seq expand size = 6459040 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.4448 value = 3.7186 dict = 1.00 all = 3.8223 } Zip/UnZip{ index = 0.0957 value = 0.2689 dict = 1.00 all = 0.2616 } ---------------------------- total value len = 2252.473540 GB avg = 0.200 KB (by entry num) total key len = 307.141911 GB avg = 0.027 KB total ukey len = 237.539691 GB avg = 0.015 KB total ukey num = 15.747909325 Billion total entry num = 11.289223273 Billion write speed all = 38.032044310 MB/sec (with seq num) write speed all = 36.690116984 MB/sec (without seq num) 2024/11/24-03:36:53.805573 139957206771264 [compaction_job.cc:1948] [default] [JOB 40913] Generated table #290589: 922720 keys, 35114338 bytes, temperature: kWarm 2024/11/24-03:36:53.805645 139957206771264 EVENT_LOG_v1 {"time_micros": 1732390613805608, "cf_name": "default", "job": 40913, "event": "table_file_creation", "file_number": 290589, "file_size": 35114338, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4288032204, "largest_seqno": 4420277834, "table_properties": {"data_size": 29146064, "index_size": 1766848, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25836160, "raw_average_key_size": 28, "raw_value_size": 108381596, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922720, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390610, "oldest_key_time": 0, "file_creation_time": 1732390610, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290589, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:36:53.846967 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290590.sst) /dev/shm/ToplingTemp/Topling-260105-y2PSk5 2024/11/24-03:36:54.842816 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290590.sst): first pass time = 1.00's, 134.785'MB/sec 2024/11/24-03:36:56.197455 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290590.sst): old prealloc_size = 147639500, real_size = 32206640 2024/11/24-03:36:56.198026 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290590.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928827 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0302 GB avg-val = 116.50 avg-zval = 32.51 2024/11/24-03:36:56.199853 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290590.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:36:57.033716 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290590.sst): second pass time = 1.35's, 79.968'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.426'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3234.138'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.328'MB/sec write SST data time = 0.13's, 282.869'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32355.537'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 79.968'MB/sec zip pipeline throughput = 79.968'MB/sec entries = 928827 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.50 avg-zval = 32.51 usrkeys = 928827 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.50 avg-zval = 32.51 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928827 bytes = 3250912 } seq expand size = 6501789 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.4443 value = 3.5837 dict = 1.00 all = 3.6960 } Zip/UnZip{ index = 0.0957 value = 0.2790 dict = 1.00 all = 0.2706 } ---------------------------- total value len = 2252.588252 GB avg = 0.200 KB (by entry num) total key len = 307.167918 GB avg = 0.027 KB total ukey len = 237.558267 GB avg = 0.015 KB total ukey num = 15.748838152 Billion total entry num = 11.290152100 Billion write speed all = 38.032286636 MB/sec (with seq num) write speed all = 36.690314128 MB/sec (without seq num) 2024/11/24-03:36:57.078696 139957206771264 [compaction_job.cc:1948] [default] [JOB 40913] Generated table #290590: 928827 keys, 36314212 bytes, temperature: kWarm 2024/11/24-03:36:57.078765 139957206771264 EVENT_LOG_v1 {"time_micros": 1732390617078728, "cf_name": "default", "job": 40913, "event": "table_file_creation", "file_number": 290590, "file_size": 36314212, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4420282836, "table_properties": {"data_size": 30195616, "index_size": 1778624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26007156, "raw_average_key_size": 28, "raw_value_size": 108210654, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928827, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390613, "oldest_key_time": 0, "file_creation_time": 1732390613, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290590, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:36:57.119218 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290591.sst) /dev/shm/ToplingTemp/Topling-260105-iRPLt5 2024/11/24-03:36:58.067636 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290591.sst): first pass time = 0.95's, 141.525'MB/sec 2024/11/24-03:36:59.325762 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290591.sst): old prealloc_size = 147639500, real_size = 27721936 2024/11/24-03:36:59.326391 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290591.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 945196 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1078 GB zip-val = 0.0257 GB avg-val = 114.00 avg-zval = 27.16 2024/11/24-03:36:59.327164 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290591.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:37:00.159171 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290591.sst): second pass time = 1.26's, 85.735'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 38.102'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2995.452'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.334'MB/sec write SST data time = 0.13's, 252.543'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.073'MB, throughput = 33783.093'MB/sec zip my value time = 1.26's, unzip length = 0.108'GB zip my value throughput = 85.735'MB/sec zip pipeline throughput = 85.735'MB/sec entries = 945196 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.00 avg-zval = 27.16 usrkeys = 945196 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.00 avg-zval = 27.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 945196 bytes = 3190048 } seq expand size = 6616372 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0257 GB dict = 1.07 MB all = 0.0318 GB } UnZip/Zip{ index = 10.4442 value = 4.1967 dict = 1.00 all = 4.2269 } Zip/UnZip{ index = 0.0957 value = 0.2383 dict = 1.00 all = 0.2366 } ---------------------------- total value len = 2252.702621 GB avg = 0.200 KB (by entry num) total key len = 307.194384 GB avg = 0.027 KB total ukey len = 237.577171 GB avg = 0.015 KB total ukey num = 15.749783348 Billion total entry num = 11.291097296 Billion write speed all = 38.032612987 MB/sec (with seq num) write speed all = 36.690590451 MB/sec (without seq num) 2024/11/24-03:37:00.198857 139957206771264 [compaction_job.cc:1948] [default] [JOB 40913] Generated table #290591: 945196 keys, 31753136 bytes, temperature: kWarm 2024/11/24-03:37:00.198930 139957206771264 EVENT_LOG_v1 {"time_micros": 1732390620198894, "cf_name": "default", "job": 40913, "event": "table_file_creation", "file_number": 290591, "file_size": 31753136, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4286255552, "largest_seqno": 4387330200, "table_properties": {"data_size": 25675456, "index_size": 1809984, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26465488, "raw_average_key_size": 28, "raw_value_size": 107752344, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 945196, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390617, "oldest_key_time": 0, "file_creation_time": 1732390617, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290591, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:37:00.234785 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290592.sst) /dev/shm/ToplingTemp/Topling-260105-etucR6 2024/11/24-03:37:01.244817 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290592.sst): first pass time = 1.01's, 132.891'MB/sec 2024/11/24-03:37:02.591208 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290592.sst): old prealloc_size = 147639500, real_size = 33162288 2024/11/24-03:37:02.591801 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290592.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927650 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.69 avg-zval = 33.58 2024/11/24-03:37:02.592633 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290592.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:37:03.445422 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290592.sst): second pass time = 1.34's, 80.483'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.624'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3187.600'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 298.029'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 23411.160'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.483'MB/sec zip pipeline throughput = 80.483'MB/sec entries = 927650 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.69 avg-zval = 33.58 usrkeys = 927650 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.69 avg-zval = 33.58 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927650 bytes = 3246784 } seq expand size = 6493550 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4420 value = 3.4745 dict = 1.00 all = 3.6019 } Zip/UnZip{ index = 0.0958 value = 0.2878 dict = 1.00 all = 0.2776 } ---------------------------- total value len = 2252.817358 GB avg = 0.200 KB (by entry num) total key len = 307.220358 GB avg = 0.027 KB total ukey len = 237.595724 GB avg = 0.015 KB total ukey num = 15.750710998 Billion total entry num = 11.292024946 Billion write speed all = 38.032846629 MB/sec (with seq num) write speed all = 36.690779361 MB/sec (without seq num) 2024/11/24-03:37:03.491197 139957206771264 [compaction_job.cc:1948] [default] [JOB 40913] Generated table #290592: 927650 keys, 37262902 bytes, temperature: kWarm 2024/11/24-03:37:03.491268 139957206771264 EVENT_LOG_v1 {"time_micros": 1732390623491230, "cf_name": "default", "job": 40913, "event": "table_file_creation", "file_number": 290592, "file_size": 37262902, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4420282657, "table_properties": {"data_size": 31153440, "index_size": 1776768, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25974200, "raw_average_key_size": 28, "raw_value_size": 108243656, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927650, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390620, "oldest_key_time": 0, "file_creation_time": 1732390620, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290592, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:37:03.527721 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290593.sst) /dev/shm/ToplingTemp/Topling-260105-0P3kd7 2024/11/24-03:37:04.573180 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290593.sst): first pass time = 1.05's, 128.387'MB/sec 2024/11/24-03:37:05.881299 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290593.sst): old prealloc_size = 147639500, real_size = 29554096 2024/11/24-03:37:05.881870 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290593.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934865 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1080 GB zip-val = 0.0275 GB avg-val = 115.57 avg-zval = 29.45 2024/11/24-03:37:05.883761 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290593.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:37:06.705246 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290593.sst): second pass time = 1.31's, 82.692'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 40.730'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3273.816'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.336'MB/sec write SST data time = 0.13's, 262.250'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31965.255'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.692'MB/sec zip pipeline throughput = 82.692'MB/sec entries = 934865 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.57 avg-zval = 29.45 usrkeys = 934865 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.57 avg-zval = 29.45 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934865 bytes = 3155184 } seq expand size = 6544055 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0275 GB dict = 1.08 MB all = 0.0336 GB } UnZip/Zip{ index = 10.4450 value = 3.9245 dict = 1.00 all = 3.9996 } Zip/UnZip{ index = 0.0957 value = 0.2548 dict = 1.00 all = 0.2500 } ---------------------------- total value len = 2252.931944 GB avg = 0.199 KB (by entry num) total key len = 307.246534 GB avg = 0.027 KB total ukey len = 237.614421 GB avg = 0.015 KB total ukey num = 15.751645863 Billion total entry num = 11.292959811 Billion write speed all = 38.033095928 MB/sec (with seq num) write speed all = 36.690982548 MB/sec (without seq num) 2024/11/24-03:37:06.748736 139957206771264 [compaction_job.cc:1948] [default] [JOB 40913] Generated table #290593: 934865 keys, 33557788 bytes, temperature: kWarm 2024/11/24-03:37:06.748810 139957206771264 EVENT_LOG_v1 {"time_micros": 1732390626748772, "cf_name": "default", "job": 40913, "event": "table_file_creation", "file_number": 290593, "file_size": 33557788, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4311634082, "largest_seqno": 4420282147, "table_properties": {"data_size": 27530144, "index_size": 1790080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26176220, "raw_average_key_size": 28, "raw_value_size": 108041582, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934865, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390623, "oldest_key_time": 0, "file_creation_time": 1732390623, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290593, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:37:06.786505 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290594.sst) /dev/shm/ToplingTemp/Topling-260105-FRyNd5 2024/11/24-03:37:07.807249 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290594.sst): first pass time = 1.02's, 131.495'MB/sec 2024/11/24-03:37:09.138928 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290594.sst): old prealloc_size = 147639500, real_size = 32805296 2024/11/24-03:37:09.139538 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290594.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931327 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 116.11 avg-zval = 33.06 2024/11/24-03:37:09.140338 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290594.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:37:10.002909 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290594.sst): second pass time = 1.33's, 81.252'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.729'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3030.536'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.316'MB/sec write SST data time = 0.13's, 291.391'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33843.652'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.252'MB/sec zip pipeline throughput = 81.252'MB/sec entries = 931327 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.11 avg-zval = 33.06 usrkeys = 931327 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.11 avg-zval = 33.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931327 bytes = 3259664 } seq expand size = 6519289 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4431 value = 3.5124 dict = 1.00 all = 3.6358 } Zip/UnZip{ index = 0.0958 value = 0.2847 dict = 1.00 all = 0.2750 } ---------------------------- total value len = 2253.046604 GB avg = 0.199 KB (by entry num) total key len = 307.272611 GB avg = 0.027 KB total ukey len = 237.633048 GB avg = 0.015 KB total ukey num = 15.752577190 Billion total entry num = 11.293891138 Billion write speed all = 38.033323456 MB/sec (with seq num) write speed all = 36.691165143 MB/sec (without seq num) 2024/11/24-03:37:10.049966 139957206771264 [compaction_job.cc:1948] [default] [JOB 40913] Generated table #290594: 931327 keys, 36915956 bytes, temperature: kWarm 2024/11/24-03:37:10.050047 139957206771264 EVENT_LOG_v1 {"time_micros": 1732390630050010, "cf_name": "default", "job": 40913, "event": "table_file_creation", "file_number": 290594, "file_size": 36915956, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679865, "largest_seqno": 4420282968, "table_properties": {"data_size": 30788640, "index_size": 1783616, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26077156, "raw_average_key_size": 28, "raw_value_size": 108140650, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931327, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390626, "oldest_key_time": 0, "file_creation_time": 1732390626, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290594, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:37:10.087821 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290595.sst) /dev/shm/ToplingTemp/Topling-260105-C94Gn9 2024/11/24-03:37:11.011953 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290595.sst): first pass time = 0.92's, 145.243'MB/sec 2024/11/24-03:37:12.320453 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290595.sst): old prealloc_size = 147639500, real_size = 30935920 2024/11/24-03:37:12.321016 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290595.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924808 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0289 GB avg-val = 117.13 avg-zval = 31.29 2024/11/24-03:37:12.322876 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290595.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:37:13.138366 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290595.sst): second pass time = 1.31's, 82.862'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.685'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3287.359'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 268.985'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 26431.061'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.862'MB/sec zip pipeline throughput = 82.862'MB/sec entries = 924808 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.13 avg-zval = 31.29 usrkeys = 924808 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.13 avg-zval = 31.29 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924808 bytes = 3121248 } seq expand size = 6473656 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0349 GB } UnZip/Zip{ index = 10.4461 value = 3.7438 dict = 1.00 all = 3.8445 } Zip/UnZip{ index = 0.0957 value = 0.2671 dict = 1.00 all = 0.2601 } ---------------------------- total value len = 2253.161401 GB avg = 0.199 KB (by entry num) total key len = 307.298506 GB avg = 0.027 KB total ukey len = 237.651544 GB avg = 0.015 KB total ukey num = 15.753501998 Billion total entry num = 11.294815946 Billion write speed all = 38.033641925 MB/sec (with seq num) write speed all = 36.691436225 MB/sec (without seq num) 2024/11/24-03:37:13.182334 139957206771264 [compaction_job.cc:1948] [default] [JOB 40913] Generated table #290595: 924808 keys, 34911342 bytes, temperature: kWarm 2024/11/24-03:37:13.182409 139957206771264 EVENT_LOG_v1 {"time_micros": 1732390633182371, "cf_name": "default", "job": 40913, "event": "table_file_creation", "file_number": 290595, "file_size": 34911342, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4326604560, "largest_seqno": 4420282157, "table_properties": {"data_size": 28933920, "index_size": 1770624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25894624, "raw_average_key_size": 28, "raw_value_size": 108323148, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924808, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390630, "oldest_key_time": 0, "file_creation_time": 1732390630, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290595, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:37:13.220236 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290596.sst) /dev/shm/ToplingTemp/Topling-260105-eOjK87 2024/11/24-03:37:14.237742 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290596.sst): first pass time = 1.02's, 131.914'MB/sec 2024/11/24-03:37:15.566048 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290596.sst): old prealloc_size = 147639500, real_size = 32908944 2024/11/24-03:37:15.566580 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290596.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929449 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.41 avg-zval = 33.25 2024/11/24-03:37:15.568650 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290596.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:37:16.428033 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290596.sst): second pass time = 1.33's, 81.503'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.944'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3523.596'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.316'MB/sec write SST data time = 0.13's, 293.406'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 31926.393'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.503'MB/sec zip pipeline throughput = 81.503'MB/sec entries = 929449 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.41 avg-zval = 33.25 usrkeys = 929449 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.41 avg-zval = 33.25 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929449 bytes = 3253088 } seq expand size = 6506143 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5122 value = 3.5005 dict = 1.00 all = 3.6261 } Zip/UnZip{ index = 0.0951 value = 0.2857 dict = 1.00 all = 0.2758 } ---------------------------- total value len = 2253.276100 GB avg = 0.199 KB (by entry num) total key len = 307.324530 GB avg = 0.027 KB total ukey len = 237.670133 GB avg = 0.015 KB total ukey num = 15.754431447 Billion total entry num = 11.295745395 Billion write speed all = 38.033873741 MB/sec (with seq num) write speed all = 36.691623180 MB/sec (without seq num) 2024/11/24-03:37:16.472627 139957206771264 [compaction_job.cc:1948] [default] [JOB 40913] Generated table #290596: 929449 keys, 37014648 bytes, temperature: kWarm 2024/11/24-03:37:16.472698 139957206771264 EVENT_LOG_v1 {"time_micros": 1732390636472662, "cf_name": "default", "job": 40913, "event": "table_file_creation", "file_number": 290596, "file_size": 37014648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679880, "largest_seqno": 4420283190, "table_properties": {"data_size": 30908096, "index_size": 1768320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26024572, "raw_average_key_size": 28, "raw_value_size": 108193190, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929449, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390633, "oldest_key_time": 0, "file_creation_time": 1732390633, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290596, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:37:16.512394 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290597.sst) /dev/shm/ToplingTemp/Topling-260105-q1UUn8 2024/11/24-03:37:17.497017 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290597.sst): first pass time = 0.98's, 136.319'MB/sec 2024/11/24-03:37:18.804628 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290597.sst): old prealloc_size = 147639500, real_size = 30776432 2024/11/24-03:37:18.805244 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290597.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924611 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0288 GB avg-val = 117.16 avg-zval = 31.11 2024/11/24-03:37:18.806147 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290597.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:37:19.612062 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290597.sst): second pass time = 1.31's, 82.893'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.041'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3003.896'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.340'MB/sec write SST data time = 0.13's, 275.547'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 35052.700'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.893'MB/sec zip pipeline throughput = 82.893'MB/sec entries = 924611 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.16 avg-zval = 31.11 usrkeys = 924611 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.16 avg-zval = 31.11 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924611 bytes = 3120576 } seq expand size = 6472277 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0288 GB dict = 1.08 MB all = 0.0348 GB } UnZip/Zip{ index = 10.3771 value = 3.7662 dict = 1.00 all = 3.8622 } Zip/UnZip{ index = 0.0964 value = 0.2655 dict = 1.00 all = 0.2589 } ---------------------------- total value len = 2253.390901 GB avg = 0.199 KB (by entry num) total key len = 307.350420 GB avg = 0.027 KB total ukey len = 237.688625 GB avg = 0.015 KB total ukey num = 15.755356058 Billion total entry num = 11.296670006 Billion write speed all = 38.034164681 MB/sec (with seq num) write speed all = 36.691867733 MB/sec (without seq num) 2024/11/24-03:37:19.655969 139957206771264 [compaction_job.cc:1948] [default] [JOB 40913] Generated table #290597: 924611 keys, 34751420 bytes, temperature: kWarm 2024/11/24-03:37:19.656055 139957206771264 EVENT_LOG_v1 {"time_micros": 1732390639656019, "cf_name": "default", "job": 40913, "event": "table_file_creation", "file_number": 290597, "file_size": 34751420, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4331112160, "largest_seqno": 4420282170, "table_properties": {"data_size": 28763104, "index_size": 1782016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25889108, "raw_average_key_size": 28, "raw_value_size": 108328730, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924611, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390636, "oldest_key_time": 0, "file_creation_time": 1732390636, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290597, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:37:19.692543 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290598.sst) /dev/shm/ToplingTemp/Topling-260105-iRu8J7 2024/11/24-03:37:20.688765 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290598.sst): first pass time = 1.00's, 134.731'MB/sec 2024/11/24-03:37:22.028533 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290598.sst): old prealloc_size = 147639500, real_size = 32964528 2024/11/24-03:37:22.029107 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290598.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926374 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 116.89 avg-zval = 33.43 2024/11/24-03:37:22.029879 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290598.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:37:22.878320 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290598.sst): second pass time = 1.34's, 80.904'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.614'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3196.007'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.12's, 296.491'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33820.897'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.904'MB/sec zip pipeline throughput = 80.904'MB/sec entries = 926374 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.89 avg-zval = 33.43 usrkeys = 926374 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.89 avg-zval = 33.43 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926374 bytes = 3242320 } seq expand size = 6484618 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5140 value = 3.4962 dict = 1.00 all = 3.6215 } Zip/UnZip{ index = 0.0951 value = 0.2860 dict = 1.00 all = 0.2761 } ---------------------------- total value len = 2253.505665 GB avg = 0.199 KB (by entry num) total key len = 307.376358 GB avg = 0.027 KB total ukey len = 237.707153 GB avg = 0.015 KB total ukey num = 15.756282432 Billion total entry num = 11.297596380 Billion write speed all = 38.034409339 MB/sec (with seq num) write speed all = 36.692067439 MB/sec (without seq num) 2024/11/24-03:37:22.924938 139957206771264 [compaction_job.cc:1948] [default] [JOB 40913] Generated table #290598: 926374 keys, 37061274 bytes, temperature: kWarm 2024/11/24-03:37:22.925024 139957206771264 EVENT_LOG_v1 {"time_micros": 1732390642924972, "cf_name": "default", "job": 40913, "event": "table_file_creation", "file_number": 290598, "file_size": 37061274, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679891, "largest_seqno": 4420282721, "table_properties": {"data_size": 30970592, "index_size": 1762176, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25938472, "raw_average_key_size": 28, "raw_value_size": 108279292, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926374, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390639, "oldest_key_time": 0, "file_creation_time": 1732390639, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290598, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:37:22.969306 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290599.sst) /dev/shm/ToplingTemp/Topling-260105-Kf4Qe5 2024/11/24-03:37:23.972865 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290599.sst): first pass time = 1.00's, 133.747'MB/sec 2024/11/24-03:37:25.263226 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290599.sst): old prealloc_size = 147639500, real_size = 30670208 2024/11/24-03:37:25.263812 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290599.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927331 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0287 GB avg-val = 116.74 avg-zval = 30.90 2024/11/24-03:37:25.264590 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290599.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:37:26.087613 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290599.sst): second pass time = 1.29's, 83.946'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.036'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3189.970'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.333'MB/sec write SST data time = 0.13's, 272.168'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 37013.051'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.946'MB/sec zip pipeline throughput = 83.946'MB/sec entries = 927331 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.74 avg-zval = 30.90 usrkeys = 927331 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.74 avg-zval = 30.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927331 bytes = 3129760 } seq expand size = 6491317 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0287 GB dict = 1.08 MB all = 0.0347 GB } UnZip/Zip{ index = 10.3778 value = 3.7783 dict = 1.00 all = 3.8732 } Zip/UnZip{ index = 0.0964 value = 0.2647 dict = 1.00 all = 0.2582 } ---------------------------- total value len = 2253.620409 GB avg = 0.199 KB (by entry num) total key len = 307.402323 GB avg = 0.027 KB total ukey len = 237.725699 GB avg = 0.015 KB total ukey num = 15.757209763 Billion total entry num = 11.298523711 Billion write speed all = 38.034686249 MB/sec (with seq num) write speed all = 36.692298150 MB/sec (without seq num) 2024/11/24-03:37:26.130312 139957206771264 [compaction_job.cc:1948] [default] [JOB 40913] Generated table #290599: 927331 keys, 34652634 bytes, temperature: kWarm 2024/11/24-03:37:26.130379 139957206771264 EVENT_LOG_v1 {"time_micros": 1732390646130343, "cf_name": "default", "job": 40913, "event": "table_file_creation", "file_number": 290599, "file_size": 34652634, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4343223170, "largest_seqno": 4420282184, "table_properties": {"data_size": 28651056, "index_size": 1787136, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25965268, "raw_average_key_size": 28, "raw_value_size": 108252466, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927331, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390642, "oldest_key_time": 0, "file_creation_time": 1732390642, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290599, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:37:26.169960 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290600.sst) /dev/shm/ToplingTemp/Topling-260105-YYpzj7 2024/11/24-03:37:26.326307 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290600.sst): first pass time = 0.16's, 247.822'MB/sec 2024/11/24-03:37:26.708308 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290600.sst): old prealloc_size = 147639500, real_size = 10906272 2024/11/24-03:37:26.708515 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290600.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 268741 prefix = 4 raw-key = 0.0043 GB zip-key = 0.0005 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0312 GB zip-val = 0.0103 GB avg-val = 116.13 avg-zval = 38.41 2024/11/24-03:37:26.708771 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290600.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0006 GB, waited 0.000 sec, Key+Value = 0.039 GB 2024/11/24-03:37:26.945029 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00/290600.sst): second pass time = 0.38's, 81.853'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.24's, 51.123'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2825.489'MB/sec (index lex order gen) rebuild zvType time = 0.20's, 0.338'MB/sec write SST data time = 0.04's, 320.621'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.318'MB, throughput = 10304.518'MB/sec zip my value time = 0.38's, unzip length = 0.031'GB zip my value throughput = 81.853'MB/sec zip pipeline throughput = 81.853'MB/sec entries = 268741 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.13 avg-zval = 38.41 usrkeys = 268741 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.13 avg-zval = 38.41 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 268741 bytes = 940608 } seq expand size = 1881187 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0054 GB value = 0.0312 GB dict = 0.32 MB all = 0.0387 GB } __ZipSize{ index = 0.0005 GB value = 0.0103 GB dict = 0.32 MB all = 0.0121 GB } UnZip/Zip{ index = 10.4208 value = 3.0232 dict = 1.00 all = 3.2011 } Zip/UnZip{ index = 0.0960 value = 0.3308 dict = 1.00 all = 0.3124 } ---------------------------- total value len = 2253.653499 GB avg = 0.199 KB (by entry num) total key len = 307.409848 GB avg = 0.027 KB total ukey len = 237.731074 GB avg = 0.015 KB total ukey num = 15.757478504 Billion total entry num = 11.298792452 Billion write speed all = 38.034805113 MB/sec (with seq num) write speed all = 36.692402179 MB/sec (without seq num) 2024/11/24-03:37:26.964740 139957206771264 [compaction_job.cc:1948] [default] [JOB 40913] Generated table #290600: 268741 keys, 12100029 bytes, temperature: kWarm 2024/11/24-03:37:26.964804 139957206771264 EVENT_LOG_v1 {"time_micros": 1732390646964771, "cf_name": "default", "job": 40913, "event": "table_file_creation", "file_number": 290600, "file_size": 12100029, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679902, "largest_seqno": 4420280958, "table_properties": {"data_size": 10323152, "index_size": 515776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 7524748, "raw_average_key_size": 28, "raw_value_size": 31208810, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 268741, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390646, "oldest_key_time": 0, "file_creation_time": 1732390646, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290600, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:37:26.979493 139957206771264 [compaction_job.cc:703] job-40913: subcompact[0], size: 0.728999 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40913/att-00 [290580,290581,290582,290583,290584,290585,290586,290587,290588,290589,290590,290591,290592,290593,290594,290595,290596,290597,290598,290599,290600] 2024/11/24-03:37:27.101616 139957206771264 (Original Log Time 2024/11/24-03:36:21.414522) EVENT_LOG_v1 {"time_micros": 1732390581414483, "job": 40913, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [290516], "files_L3": [290576, 290577, 290534, 290477, 290478, 290479, 290480, 290481, 290482, 290483, 290484, 290485, 290486, 290487, 290488, 290489, 290453, 290454, 290455, 290456, 290457, 290458, 290549], "score": 0.968504, "input_data_size": 748057161, "oldest_snapshot_seqno": 4420697028} 2024/11/24-03:37:27.101619 139957206771264 (Original Log Time 2024/11/24-03:37:27.100037) [compaction_job.cc:2022] [default] [JOB 40913] Compacted 1@2 + 23@3 files to L3 => 728999135 bytes 2024/11/24-03:37:27.101621 139957206771264 (Original Log Time 2024/11/24-03:37:27.101545) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 21 0 0 0] max score 0.03, MB/sec: 11.4 rd, 11.1 wr, level 3, files in(1, 23) out(21 +0 blob) MB in(27.1, 686.3 +0.0 blob) out(695.2 +0.0 blob), read-write-amplify(52.0) write-amplify(25.6) OK, records in: 18919276, records dropped: 101935 output_compression: Snappy 2024/11/24-03:37:27.101624 139957206771264 (Original Log Time 2024/11/24-03:37:27.101590) EVENT_LOG_v1 {"time_micros": 1732390647101567, "cf": "default", "job": 40913, "event": "compaction_finished", "compaction_time_micros": 65564979, "compaction_time_cpu_micros": 28106168, "output_level": 3, "num_output_files": 21, "total_output_size": 728999135, "num_input_records": 18919276, "num_output_records": 18817341, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 504143657, "file_range_sync_nanos": 0, "file_fsync_nanos": 14602, "file_prepare_write_nanos": 60882, "lsm_state": [0, 0, 0, 21, 0, 0, 0]} 2024/11/24-03:37:27.246677 139957206771264 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732390581,"executesMs":65825,"compactionJobId":40913,"attempt":0,"compactionInputRawBytes":2737564356,"compactionInputZipBytes":748057161,"compactionOutputRawBytes":0,"compactionOutputZipBytes":728999135,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-41a59eed9fe1b45f9889c823d2a0b466-092d8247a6c03dbf-00"}}, response = 2024/11/24-03:37:27.246750 139957206771264 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-40913/att-00: olev 3, work 65.825 s, result 7.623 ms, install 1.603 ms, input{raw 2.550 GiB zip 713.403 MiB}