2024/11/24-03:56:58.056314 139957584442944 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00: fork to child time = 0.008273 sec 2024/11/24-03:56:58.057083 139957584442944 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00 2024/11/24-03:56:58.062443 139957584442944 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00 2024/11/24-03:56:58.063332 139957584442944 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-03:56:58.063553 139957584442944 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-03:56:58.111764 139957584442944 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41143/att-00/MANIFEST-206347 2024/11/24-03:56:58.133224 139957584442944 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41143/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 291117, last_sequence is 4428111048, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 290925 2024/11/24-03:56:58.133231 139957584442944 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-03:56:58.133363 139957584442944 [version_set.cc:5721] Creating manifest 291117 2024/11/24-03:56:58.167389 139957584442944 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-03:56:58.189094 139957584442944 [compaction_job.cc:2437] [default] [JOB 41143] Compacting 1@2 + 21@3 files to L3, score 0.97, subcompactions 1 : 1 2024/11/24-03:56:58.189111 139957584442944 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 30818 Base level 2, inputs: [291058(27MB)], [291114(33MB) 291115(987KB) 291014(972KB) 290973(34MB) 290974(35MB) 290975(34MB) 290976(33MB) 291015(35MB) 291016(35MB) 291017(34MB) 291018(33MB) 291019(35MB) 291020(35MB) 291021(31MB) 291022(32MB) 291023(33MB) 291024(30MB) 291025(34MB) 291026(31MB) 291027(33MB) 291089(32MB)] 2024/11/24-03:56:58.196038 139957584442944 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-03:56:58.196062 139957584442944 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-03:56:58.196077 139957584442944 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-03:56:58.196082 139957584442944 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-03:56:58.196093 139957584442944 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-03:56:58.741450 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291118.sst) /dev/shm/ToplingTemp/Topling-263632-zO8mjM 2024/11/24-03:56:59.291493 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291118.sst): first pass time = 0.55's, 244.033'MB/sec 2024/11/24-03:57:00.886952 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291118.sst): old prealloc_size = 147639500, real_size = 31195152 2024/11/24-03:57:00.887545 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291118.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923391 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0292 GB avg-val = 117.35 avg-zval = 31.62 2024/11/24-03:57:00.888344 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291118.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:57:01.703252 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291118.sst): second pass time = 1.59's, 67.959'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.083'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3185.531'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 277.433'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 30378.040'MB/sec zip my value time = 1.59's, unzip length = 0.108'GB zip my value throughput = 67.959'MB/sec zip pipeline throughput = 112.378'MB/sec entries = 923391 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.35 avg-zval = 31.62 usrkeys = 923391 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.35 avg-zval = 31.62 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923391 bytes = 3116464 } seq expand size = 6463737 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0352 GB } UnZip/Zip{ index = 10.4437 value = 3.7116 dict = 1.00 all = 3.8165 } Zip/UnZip{ index = 0.0958 value = 0.2694 dict = 1.00 all = 0.2620 } ---------------------------- total value len = 2251.440400 GB avg = 0.200 KB (by entry num) total key len = 306.908817 GB avg = 0.027 KB total ukey len = 237.373195 GB avg = 0.015 KB total ukey num = 15.739584554 Billion total entry num = 11.280898502 Billion write speed all = 37.342991022 MB/sec (with seq num) write speed all = 36.025696296 MB/sec (without seq num) 2024/11/24-03:57:01.748772 139957584442944 [compaction_job.cc:1948] [default] [JOB 41143] Generated table #291118: 923391 keys, 35167574 bytes, temperature: kWarm 2024/11/24-03:57:01.748870 139957584442944 EVENT_LOG_v1 {"time_micros": 1732391821748811, "cf_name": "default", "job": 41143, "event": "table_file_creation", "file_number": 291118, "file_size": 35167574, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4299628100, "largest_seqno": 4427728600, "table_properties": {"data_size": 29195776, "index_size": 1768320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25854948, "raw_average_key_size": 28, "raw_value_size": 108362866, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923391, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391818, "oldest_key_time": 0, "file_creation_time": 1732391818, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291118, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:57:01.787659 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291119.sst) /dev/shm/ToplingTemp/Topling-263632-vrckzN 2024/11/24-03:57:02.727487 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291119.sst): first pass time = 0.94's, 142.818'MB/sec 2024/11/24-03:57:04.272966 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291119.sst): old prealloc_size = 147639500, real_size = 32529600 2024/11/24-03:57:04.273551 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291119.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922642 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0305 GB avg-val = 117.47 avg-zval = 33.10 2024/11/24-03:57:04.274358 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291119.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:57:05.102072 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291119.sst): second pass time = 1.54's, 70.187'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 44.166'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3191.291'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.328'MB/sec write SST data time = 0.13's, 291.392'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 26283.162'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.187'MB/sec zip pipeline throughput = 105.969'MB/sec entries = 922642 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.47 avg-zval = 33.10 usrkeys = 922642 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.47 avg-zval = 33.10 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922642 bytes = 3229264 } seq expand size = 6458494 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5125 value = 3.5485 dict = 1.00 all = 3.6655 } Zip/UnZip{ index = 0.0951 value = 0.2818 dict = 1.00 all = 0.2728 } ---------------------------- total value len = 2251.555242 GB avg = 0.200 KB (by entry num) total key len = 306.934651 GB avg = 0.027 KB total ukey len = 237.391648 GB avg = 0.015 KB total ukey num = 15.740507196 Billion total entry num = 11.281821144 Billion write speed all = 37.343191773 MB/sec (with seq num) write speed all = 36.025854662 MB/sec (without seq num) 2024/11/24-03:57:05.148495 139957584442944 [compaction_job.cc:1948] [default] [JOB 41143] Generated table #291119: 922642 keys, 36616872 bytes, temperature: kWarm 2024/11/24-03:57:05.148565 139957584442944 EVENT_LOG_v1 {"time_micros": 1732391825148529, "cf_name": "default", "job": 41143, "event": "table_file_creation", "file_number": 291119, "file_size": 36616872, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178686, "largest_seqno": 4427728738, "table_properties": {"data_size": 30543408, "index_size": 1755328, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25833976, "raw_average_key_size": 28, "raw_value_size": 108383820, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922642, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391821, "oldest_key_time": 0, "file_creation_time": 1732391821, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291119, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:57:05.185051 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291120.sst) /dev/shm/ToplingTemp/Topling-263632-JN6hZM 2024/11/24-03:57:06.105201 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291120.sst): first pass time = 0.92's, 145.872'MB/sec 2024/11/24-03:57:07.630471 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291120.sst): old prealloc_size = 147639500, real_size = 33452544 2024/11/24-03:57:07.631054 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291120.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926894 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 116.80 avg-zval = 33.94 2024/11/24-03:57:07.631839 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291120.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:57:08.486443 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291120.sst): second pass time = 1.52's, 71.018'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.871'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3135.072'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.12's, 302.272'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32775.078'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 71.018'MB/sec zip pipeline throughput = 90.991'MB/sec entries = 926894 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.80 avg-zval = 33.94 usrkeys = 926894 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.80 avg-zval = 33.94 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926894 bytes = 3244144 } seq expand size = 6488258 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5122 value = 3.4417 dict = 1.00 all = 3.5743 } Zip/UnZip{ index = 0.0951 value = 0.2906 dict = 1.00 all = 0.2798 } ---------------------------- total value len = 2251.669995 GB avg = 0.200 KB (by entry num) total key len = 306.960604 GB avg = 0.027 KB total ukey len = 237.410186 GB avg = 0.015 KB total ukey num = 15.741434090 Billion total entry num = 11.282748038 Billion write speed all = 37.343400812 MB/sec (with seq num) write speed all = 36.026020547 MB/sec (without seq num) 2024/11/24-03:57:08.532148 139957584442944 [compaction_job.cc:1948] [default] [JOB 41143] Generated table #291120: 926894 keys, 37551296 bytes, temperature: kWarm 2024/11/24-03:57:08.532221 139957584442944 EVENT_LOG_v1 {"time_micros": 1732391828532183, "cf_name": "default", "job": 41143, "event": "table_file_creation", "file_number": 291120, "file_size": 37551296, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178700, "largest_seqno": 4427728617, "table_properties": {"data_size": 31457200, "index_size": 1763456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25953032, "raw_average_key_size": 28, "raw_value_size": 108264728, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926894, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391825, "oldest_key_time": 0, "file_creation_time": 1732391825, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291120, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:57:08.567247 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291121.sst) /dev/shm/ToplingTemp/Topling-263632-7XPXCN 2024/11/24-03:57:09.557313 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291121.sst): first pass time = 0.99's, 135.570'MB/sec 2024/11/24-03:57:11.078508 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291121.sst): old prealloc_size = 147639500, real_size = 32038576 2024/11/24-03:57:11.079084 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291121.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926765 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0300 GB avg-val = 116.82 avg-zval = 32.42 2024/11/24-03:57:11.079857 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291121.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:57:11.911686 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291121.sst): second pass time = 1.52's, 71.213'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.374'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3192.479'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 288.892'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33586.955'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 71.213'MB/sec zip pipeline throughput = 107.265'MB/sec entries = 926765 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.82 avg-zval = 32.42 usrkeys = 926765 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.82 avg-zval = 32.42 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926765 bytes = 3243696 } seq expand size = 6487355 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.5131 value = 3.6037 dict = 1.00 all = 3.7142 } Zip/UnZip{ index = 0.0951 value = 0.2775 dict = 1.00 all = 0.2692 } ---------------------------- total value len = 2251.784751 GB avg = 0.200 KB (by entry num) total key len = 306.986554 GB avg = 0.027 KB total ukey len = 237.428721 GB avg = 0.015 KB total ukey num = 15.742360855 Billion total entry num = 11.283674803 Billion write speed all = 37.343587546 MB/sec (with seq num) write speed all = 36.026164931 MB/sec (without seq num) 2024/11/24-03:57:11.956663 139957584442944 [compaction_job.cc:1948] [default] [JOB 41143] Generated table #291121: 926765 keys, 36136758 bytes, temperature: kWarm 2024/11/24-03:57:11.956735 139957584442944 EVENT_LOG_v1 {"time_micros": 1732391831956697, "cf_name": "default", "job": 41143, "event": "table_file_creation", "file_number": 291121, "file_size": 36136758, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178719, "largest_seqno": 4427728632, "table_properties": {"data_size": 30043616, "index_size": 1763072, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25949420, "raw_average_key_size": 28, "raw_value_size": 108268322, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926765, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391828, "oldest_key_time": 0, "file_creation_time": 1732391828, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291121, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:57:11.993853 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291122.sst) /dev/shm/ToplingTemp/Topling-263632-8VsYZP 2024/11/24-03:57:13.016713 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291122.sst): first pass time = 1.02's, 131.223'MB/sec 2024/11/24-03:57:14.343064 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291122.sst): old prealloc_size = 147639500, real_size = 31473296 2024/11/24-03:57:14.343634 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291122.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919893 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 117.91 avg-zval = 32.05 2024/11/24-03:57:14.344398 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291122.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:57:15.162110 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291122.sst): second pass time = 1.32's, 81.860'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.407'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3230.637'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.13's, 280.399'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 32726.368'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.860'MB/sec zip pipeline throughput = 81.860'MB/sec entries = 919893 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.91 avg-zval = 32.05 usrkeys = 919893 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.91 avg-zval = 32.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919893 bytes = 3219648 } seq expand size = 6439251 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.08 MB all = 0.0356 GB } UnZip/Zip{ index = 10.4431 value = 3.6790 dict = 1.00 all = 3.7753 } Zip/UnZip{ index = 0.0958 value = 0.2718 dict = 1.00 all = 0.2649 } ---------------------------- total value len = 2251.899651 GB avg = 0.200 KB (by entry num) total key len = 307.012311 GB avg = 0.027 KB total ukey len = 237.447119 GB avg = 0.015 KB total ukey num = 15.743280748 Billion total entry num = 11.284594696 Billion write speed all = 37.343868829 MB/sec (with seq num) write speed all = 36.026401310 MB/sec (without seq num) 2024/11/24-03:57:15.205814 139957584442944 [compaction_job.cc:1948] [default] [JOB 41143] Generated table #291122: 919893 keys, 35551610 bytes, temperature: kWarm 2024/11/24-03:57:15.205879 139957584442944 EVENT_LOG_v1 {"time_micros": 1732391835205844, "cf_name": "default", "job": 41143, "event": "table_file_creation", "file_number": 291122, "file_size": 35551610, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4278678362, "largest_seqno": 4427728649, "table_properties": {"data_size": 29481408, "index_size": 1761728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25757004, "raw_average_key_size": 28, "raw_value_size": 108460746, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 919893, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391831, "oldest_key_time": 0, "file_creation_time": 1732391831, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291122, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:57:15.260529 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291123.sst) /dev/shm/ToplingTemp/Topling-263632-g3asON 2024/11/24-03:57:16.284041 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291123.sst): first pass time = 1.02's, 131.140'MB/sec 2024/11/24-03:57:17.607894 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291123.sst): old prealloc_size = 147639500, real_size = 33082928 2024/11/24-03:57:17.608444 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291123.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926501 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 116.87 avg-zval = 33.55 2024/11/24-03:57:17.609220 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291123.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:57:18.457674 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291123.sst): second pass time = 1.32's, 81.839'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.754'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3343.062'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.12's, 300.019'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32664.247'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.839'MB/sec zip pipeline throughput = 81.839'MB/sec entries = 926501 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.87 avg-zval = 33.55 usrkeys = 926501 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.87 avg-zval = 33.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926501 bytes = 3242768 } seq expand size = 6485507 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5139 value = 3.4828 dict = 1.00 all = 3.6099 } Zip/UnZip{ index = 0.0951 value = 0.2871 dict = 1.00 all = 0.2770 } ---------------------------- total value len = 2252.014412 GB avg = 0.200 KB (by entry num) total key len = 307.038253 GB avg = 0.027 KB total ukey len = 237.465649 GB avg = 0.015 KB total ukey num = 15.744207249 Billion total entry num = 11.285521197 Billion write speed all = 37.344126167 MB/sec (with seq num) write speed all = 36.026613845 MB/sec (without seq num) 2024/11/24-03:57:18.503353 139957584442944 [compaction_job.cc:1948] [default] [JOB 41143] Generated table #291123: 926501 keys, 37180094 bytes, temperature: kWarm 2024/11/24-03:57:18.503426 139957584442944 EVENT_LOG_v1 {"time_micros": 1732391838503387, "cf_name": "default", "job": 41143, "event": "table_file_creation", "file_number": 291123, "file_size": 37180094, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178730, "largest_seqno": 4427728836, "table_properties": {"data_size": 31088672, "index_size": 1762432, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25942028, "raw_average_key_size": 28, "raw_value_size": 108275770, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926501, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391835, "oldest_key_time": 0, "file_creation_time": 1732391835, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291123, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:57:18.540453 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291124.sst) /dev/shm/ToplingTemp/Topling-263632-6PxTbN 2024/11/24-03:57:19.565833 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291124.sst): first pass time = 1.03's, 130.901'MB/sec 2024/11/24-03:57:20.930640 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291124.sst): old prealloc_size = 147639500, real_size = 33257008 2024/11/24-03:57:20.931177 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291124.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926774 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.82 avg-zval = 33.73 2024/11/24-03:57:20.931959 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291124.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:57:21.781025 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291124.sst): second pass time = 1.36's, 79.376'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.929'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3435.113'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.12's, 301.196'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32662.596'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.376'MB/sec zip pipeline throughput = 79.376'MB/sec entries = 926774 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.82 avg-zval = 33.73 usrkeys = 926774 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.82 avg-zval = 33.73 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926774 bytes = 3243728 } seq expand size = 6487418 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5132 value = 3.4632 dict = 1.00 all = 3.5930 } Zip/UnZip{ index = 0.0951 value = 0.2887 dict = 1.00 all = 0.2783 } ---------------------------- total value len = 2252.129168 GB avg = 0.200 KB (by entry num) total key len = 307.064203 GB avg = 0.027 KB total ukey len = 237.484185 GB avg = 0.015 KB total ukey num = 15.745134023 Billion total entry num = 11.286447971 Billion write speed all = 37.344368367 MB/sec (with seq num) write speed all = 36.026811747 MB/sec (without seq num) 2024/11/24-03:57:21.826685 139957584442944 [compaction_job.cc:1948] [default] [JOB 41143] Generated table #291124: 926774 keys, 37355342 bytes, temperature: kWarm 2024/11/24-03:57:21.826754 139957584442944 EVENT_LOG_v1 {"time_micros": 1732391841826720, "cf_name": "default", "job": 41143, "event": "table_file_creation", "file_number": 291124, "file_size": 37355342, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680186, "largest_seqno": 4427728665, "table_properties": {"data_size": 31262048, "index_size": 1763072, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25949672, "raw_average_key_size": 28, "raw_value_size": 108268148, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926774, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391838, "oldest_key_time": 0, "file_creation_time": 1732391838, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291124, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:57:21.869549 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291125.sst) /dev/shm/ToplingTemp/Topling-263632-EEUJRQ 2024/11/24-03:57:22.873656 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291125.sst): first pass time = 1.00's, 133.674'MB/sec 2024/11/24-03:57:24.170055 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291125.sst): old prealloc_size = 147639500, real_size = 30775472 2024/11/24-03:57:24.170629 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291125.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926605 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0288 GB avg-val = 116.85 avg-zval = 31.05 2024/11/24-03:57:24.171386 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291125.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:57:24.983141 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291125.sst): second pass time = 1.30's, 83.568'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 42.747'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3217.755'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.12's, 280.320'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32828.873'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.568'MB/sec zip pipeline throughput = 83.568'MB/sec entries = 926605 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.85 avg-zval = 31.05 usrkeys = 926605 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.85 avg-zval = 31.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926605 bytes = 3127312 } seq expand size = 6486235 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0288 GB dict = 1.08 MB all = 0.0348 GB } UnZip/Zip{ index = 10.4464 value = 3.7634 dict = 1.00 all = 3.8617 } Zip/UnZip{ index = 0.0957 value = 0.2657 dict = 1.00 all = 0.2590 } ---------------------------- total value len = 2252.243927 GB avg = 0.200 KB (by entry num) total key len = 307.090148 GB avg = 0.027 KB total ukey len = 237.502717 GB avg = 0.015 KB total ukey num = 15.746060628 Billion total entry num = 11.287374576 Billion write speed all = 37.344676579 MB/sec (with seq num) write speed all = 36.027073356 MB/sec (without seq num) 2024/11/24-03:57:25.026223 139957584442944 [compaction_job.cc:1948] [default] [JOB 41143] Generated table #291125: 926605 keys, 34755780 bytes, temperature: kWarm 2024/11/24-03:57:25.026301 139957584442944 EVENT_LOG_v1 {"time_micros": 1732391845026266, "cf_name": "default", "job": 41143, "event": "table_file_creation", "file_number": 291125, "file_size": 34755780, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4318927245, "largest_seqno": 4427728682, "table_properties": {"data_size": 28769632, "index_size": 1774016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25944940, "raw_average_key_size": 28, "raw_value_size": 108272862, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926605, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391841, "oldest_key_time": 0, "file_creation_time": 1732391841, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291125, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:57:25.062487 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291126.sst) /dev/shm/ToplingTemp/Topling-263632-WL7izO 2024/11/24-03:57:25.902016 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291126.sst): first pass time = 0.84's, 159.880'MB/sec 2024/11/24-03:57:27.219693 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291126.sst): old prealloc_size = 147639500, real_size = 31585232 2024/11/24-03:57:27.220276 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291126.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919587 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1085 GB zip-val = 0.0296 GB avg-val = 117.95 avg-zval = 32.19 2024/11/24-03:57:27.221026 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291126.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:57:28.031453 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291126.sst): second pass time = 1.32's, 82.367'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.936'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3093.885'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 284.378'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 34574.078'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.367'MB/sec zip pipeline throughput = 82.367'MB/sec entries = 919587 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.95 avg-zval = 32.19 usrkeys = 919587 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.95 avg-zval = 32.19 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919587 bytes = 3218576 } seq expand size = 6437109 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0296 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5118 value = 3.6638 dict = 1.00 all = 3.7633 } Zip/UnZip{ index = 0.0951 value = 0.2729 dict = 1.00 all = 0.2657 } ---------------------------- total value len = 2252.358833 GB avg = 0.200 KB (by entry num) total key len = 307.115896 GB avg = 0.027 KB total ukey len = 237.521108 GB avg = 0.015 KB total ukey num = 15.746980215 Billion total entry num = 11.288294163 Billion write speed all = 37.345067861 MB/sec (with seq num) write speed all = 36.027415901 MB/sec (without seq num) 2024/11/24-03:57:28.076478 139957584442944 [compaction_job.cc:1948] [default] [JOB 41143] Generated table #291126: 919587 keys, 35664564 bytes, temperature: kWarm 2024/11/24-03:57:28.076550 139957584442944 EVENT_LOG_v1 {"time_micros": 1732391848076514, "cf_name": "default", "job": 41143, "event": "table_file_creation", "file_number": 291126, "file_size": 35664564, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680199, "largest_seqno": 4427727906, "table_properties": {"data_size": 29605504, "index_size": 1749632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25748436, "raw_average_key_size": 28, "raw_value_size": 108469390, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 919587, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391845, "oldest_key_time": 0, "file_creation_time": 1732391845, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291126, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:57:28.114944 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291127.sst) /dev/shm/ToplingTemp/Topling-263632-cDNeYN 2024/11/24-03:57:28.968615 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291127.sst): first pass time = 0.85's, 157.242'MB/sec 2024/11/24-03:57:30.308159 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291127.sst): old prealloc_size = 147639500, real_size = 33319712 2024/11/24-03:57:30.308724 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291127.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926557 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.86 avg-zval = 33.81 2024/11/24-03:57:30.310503 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291127.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:57:31.164075 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291127.sst): second pass time = 1.34's, 80.906'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.717'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3252.548'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 298.872'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32739.937'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.906'MB/sec zip pipeline throughput = 80.906'MB/sec entries = 926557 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.86 avg-zval = 33.81 usrkeys = 926557 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.86 avg-zval = 33.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926557 bytes = 3242960 } seq expand size = 6485899 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5145 value = 3.4564 dict = 1.00 all = 3.5871 } Zip/UnZip{ index = 0.0951 value = 0.2893 dict = 1.00 all = 0.2788 } ---------------------------- total value len = 2252.473593 GB avg = 0.200 KB (by entry num) total key len = 307.141840 GB avg = 0.027 KB total ukey len = 237.539640 GB avg = 0.015 KB total ukey num = 15.747906772 Billion total entry num = 11.289220720 Billion write speed all = 37.345413878 MB/sec (with seq num) write speed all = 36.027713994 MB/sec (without seq num) 2024/11/24-03:57:31.211734 139957584442944 [compaction_job.cc:1948] [default] [JOB 41143] Generated table #291127: 926557 keys, 37417004 bytes, temperature: kWarm 2024/11/24-03:57:31.211797 139957584442944 EVENT_LOG_v1 {"time_micros": 1732391851211760, "cf_name": "default", "job": 41143, "event": "table_file_creation", "file_number": 291127, "file_size": 37417004, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680212, "largest_seqno": 4427728700, "table_properties": {"data_size": 31325456, "index_size": 1762432, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25943596, "raw_average_key_size": 28, "raw_value_size": 108274134, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926557, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391848, "oldest_key_time": 0, "file_creation_time": 1732391848, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291127, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:57:31.247771 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291128.sst) /dev/shm/ToplingTemp/Topling-263632-m0MbQP 2024/11/24-03:57:32.289072 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291128.sst): first pass time = 1.04's, 128.899'MB/sec 2024/11/24-03:57:33.634744 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291128.sst): old prealloc_size = 147639500, real_size = 33289264 2024/11/24-03:57:33.635351 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291128.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926938 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.80 avg-zval = 33.75 2024/11/24-03:57:33.636143 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291128.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:57:34.475715 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291128.sst): second pass time = 1.34's, 80.529'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.464'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3044.117'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.12's, 299.296'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32850.122'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.529'MB/sec zip pipeline throughput = 80.529'MB/sec entries = 926938 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.80 avg-zval = 33.75 usrkeys = 926938 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.80 avg-zval = 33.75 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926938 bytes = 3244304 } seq expand size = 6488566 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4441 value = 3.4608 dict = 1.00 all = 3.5895 } Zip/UnZip{ index = 0.0957 value = 0.2889 dict = 1.00 all = 0.2786 } ---------------------------- total value len = 2252.588345 GB avg = 0.200 KB (by entry num) total key len = 307.167794 GB avg = 0.027 KB total ukey len = 237.558178 GB avg = 0.015 KB total ukey num = 15.748833710 Billion total entry num = 11.290147658 Billion write speed all = 37.345662364 MB/sec (with seq num) write speed all = 36.027917955 MB/sec (without seq num) 2024/11/24-03:57:34.526642 139957584442944 [compaction_job.cc:1948] [default] [JOB 41143] Generated table #291128: 926938 keys, 37391488 bytes, temperature: kWarm 2024/11/24-03:57:34.526714 139957584442944 EVENT_LOG_v1 {"time_micros": 1732391854526677, "cf_name": "default", "job": 41143, "event": "table_file_creation", "file_number": 291128, "file_size": 37391488, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4427728719, "table_properties": {"data_size": 31282336, "index_size": 1775040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25954264, "raw_average_key_size": 28, "raw_value_size": 108263468, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926938, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391851, "oldest_key_time": 0, "file_creation_time": 1732391851, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291128, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:57:34.562537 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291129.sst) /dev/shm/ToplingTemp/Topling-263632-p6OvHO 2024/11/24-03:57:35.438721 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291129.sst): first pass time = 0.88's, 153.191'MB/sec 2024/11/24-03:57:36.740675 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291129.sst): old prealloc_size = 147639500, real_size = 29423616 2024/11/24-03:57:36.741249 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291129.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 935658 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1080 GB zip-val = 0.0274 GB avg-val = 115.45 avg-zval = 29.28 2024/11/24-03:57:36.742025 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291129.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:57:37.559987 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291129.sst): second pass time = 1.30's, 83.013'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 40.946'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3260.489'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.337'MB/sec write SST data time = 0.13's, 267.300'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33500.404'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.013'MB/sec zip pipeline throughput = 83.013'MB/sec entries = 935658 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.45 avg-zval = 29.28 usrkeys = 935658 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.45 avg-zval = 29.28 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 935658 bytes = 3274816 } seq expand size = 6549606 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0274 GB dict = 1.08 MB all = 0.0335 GB } UnZip/Zip{ index = 10.4448 value = 3.9426 dict = 1.00 all = 4.0010 } Zip/UnZip{ index = 0.0957 value = 0.2536 dict = 1.00 all = 0.2499 } ---------------------------- total value len = 2252.702914 GB avg = 0.200 KB (by entry num) total key len = 307.193992 GB avg = 0.027 KB total ukey len = 237.576891 GB avg = 0.015 KB total ukey num = 15.749769368 Billion total entry num = 11.291083316 Billion write speed all = 37.346035600 MB/sec (with seq num) write speed all = 36.028241283 MB/sec (without seq num) 2024/11/24-03:57:37.604882 139957584442944 [compaction_job.cc:1948] [default] [JOB 41143] Generated table #291129: 935658 keys, 33546204 bytes, temperature: kWarm 2024/11/24-03:57:37.604953 139957584442944 EVENT_LOG_v1 {"time_micros": 1732391857604916, "cf_name": "default", "job": 41143, "event": "table_file_creation", "file_number": 291129, "file_size": 33546204, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680463, "largest_seqno": 4427728075, "table_properties": {"data_size": 27397872, "index_size": 1791616, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26198424, "raw_average_key_size": 28, "raw_value_size": 108019364, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 935658, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391854, "oldest_key_time": 0, "file_creation_time": 1732391854, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291129, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:57:37.642116 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291130.sst) /dev/shm/ToplingTemp/Topling-263632-TX8dYO 2024/11/24-03:57:38.604371 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291130.sst): first pass time = 0.96's, 139.488'MB/sec 2024/11/24-03:57:39.867708 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291130.sst): old prealloc_size = 147639500, real_size = 28251712 2024/11/24-03:57:39.868296 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291130.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 944492 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1078 GB zip-val = 0.0262 GB avg-val = 114.11 avg-zval = 27.76 2024/11/24-03:57:39.869072 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291130.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:57:40.706903 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291130.sst): second pass time = 1.26's, 85.399'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 38.610'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3176.146'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.332'MB/sec write SST data time = 0.13's, 253.937'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.073'MB, throughput = 35213.056'MB/sec zip my value time = 1.26's, unzip length = 0.108'GB zip my value throughput = 85.399'MB/sec zip pipeline throughput = 85.399'MB/sec entries = 944492 avg-key = 28.00 avg-zkey = 1.90 avg-val = 114.11 avg-zval = 27.76 usrkeys = 944492 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.11 avg-zval = 27.76 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 944492 bytes = 3305744 } seq expand size = 6611444 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0262 GB dict = 1.07 MB all = 0.0324 GB } UnZip/Zip{ index = 10.5134 value = 4.1105 dict = 1.00 all = 4.1424 } Zip/UnZip{ index = 0.0951 value = 0.2433 dict = 1.00 all = 0.2414 } ---------------------------- total value len = 2252.817298 GB avg = 0.200 KB (by entry num) total key len = 307.220438 GB avg = 0.027 KB total ukey len = 237.595781 GB avg = 0.015 KB total ukey num = 15.750713860 Billion total entry num = 11.292027808 Billion write speed all = 37.346375561 MB/sec (with seq num) write speed all = 36.028531514 MB/sec (without seq num) 2024/11/24-03:57:40.750010 139957584442944 [compaction_job.cc:1948] [default] [JOB 41143] Generated table #291130: 944492 keys, 32400718 bytes, temperature: kWarm 2024/11/24-03:57:40.750078 139957584442944 EVENT_LOG_v1 {"time_micros": 1732391860750043, "cf_name": "default", "job": 41143, "event": "table_file_creation", "file_number": 291130, "file_size": 32400718, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4427728030, "table_properties": {"data_size": 26218672, "index_size": 1796736, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26445776, "raw_average_key_size": 28, "raw_value_size": 107771956, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 944492, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391857, "oldest_key_time": 0, "file_creation_time": 1732391857, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291130, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:57:40.785721 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291131.sst) /dev/shm/ToplingTemp/Topling-263632-55hMPO 2024/11/24-03:57:41.775775 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291131.sst): first pass time = 0.99's, 135.571'MB/sec 2024/11/24-03:57:43.073364 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291131.sst): old prealloc_size = 147639500, real_size = 31464016 2024/11/24-03:57:43.073895 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291131.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.06 2024/11/24-03:57:43.074675 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291131.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:57:43.883166 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291131.sst): second pass time = 1.30's, 83.652'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.890'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3482.911'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.336'MB/sec write SST data time = 0.12's, 285.779'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 30840.795'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.652'MB/sec zip pipeline throughput = 83.652'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.06 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4439 value = 3.6805 dict = 1.00 all = 3.7765 } Zip/UnZip{ index = 0.0957 value = 0.2717 dict = 1.00 all = 0.2648 } ---------------------------- total value len = 2252.932210 GB avg = 0.199 KB (by entry num) total key len = 307.246178 GB avg = 0.027 KB total ukey len = 237.614167 GB avg = 0.015 KB total ukey num = 15.751633160 Billion total entry num = 11.292947108 Billion write speed all = 37.346696940 MB/sec (with seq num) write speed all = 36.028806671 MB/sec (without seq num) 2024/11/24-03:57:43.929260 139957584442944 [compaction_job.cc:1948] [default] [JOB 41143] Generated table #291131: 919300 keys, 35540722 bytes, temperature: kWarm 2024/11/24-03:57:43.929327 139957584442944 EVENT_LOG_v1 {"time_micros": 1732391863929292, "cf_name": "default", "job": 41143, "event": "table_file_creation", "file_number": 291131, "file_size": 35540722, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4269108536, "largest_seqno": 4412879619, "table_properties": {"data_size": 29473536, "index_size": 1760448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391860, "oldest_key_time": 0, "file_creation_time": 1732391860, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291131, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:57:43.965815 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291132.sst) /dev/shm/ToplingTemp/Topling-263632-bSo27M 2024/11/24-03:57:44.839124 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291132.sst): first pass time = 0.87's, 153.694'MB/sec 2024/11/24-03:57:46.087742 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291132.sst): old prealloc_size = 147639500, real_size = 28251232 2024/11/24-03:57:46.088312 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291132.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 942629 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1078 GB zip-val = 0.0262 GB avg-val = 114.39 avg-zval = 27.81 2024/11/24-03:57:46.089060 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291132.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:57:46.920410 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291132.sst): second pass time = 1.25's, 86.410'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 38.764'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3301.615'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.334'MB/sec write SST data time = 0.13's, 255.532'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.075'MB, throughput = 34080.309'MB/sec zip my value time = 1.25's, unzip length = 0.108'GB zip my value throughput = 86.410'MB/sec zip pipeline throughput = 86.410'MB/sec entries = 942629 avg-key = 28.00 avg-zkey = 1.92 avg-val = 114.39 avg-zval = 27.81 usrkeys = 942629 avg-key = 20.00 avg-zkey = 1.92 avg-val = 121.39 avg-zval = 27.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 942629 bytes = 3181392 } seq expand size = 6598403 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0262 GB dict = 1.08 MB all = 0.0323 GB } UnZip/Zip{ index = 10.4425 value = 4.1139 dict = 1.00 all = 4.1584 } Zip/UnZip{ index = 0.0958 value = 0.2431 dict = 1.00 all = 0.2405 } ---------------------------- total value len = 2253.046633 GB avg = 0.199 KB (by entry num) total key len = 307.272572 GB avg = 0.027 KB total ukey len = 237.633020 GB avg = 0.015 KB total ukey num = 15.752575789 Billion total entry num = 11.293889737 Billion write speed all = 37.347096404 MB/sec (with seq num) write speed all = 36.029154522 MB/sec (without seq num) 2024/11/24-03:57:46.963038 139957584442944 [compaction_job.cc:1948] [default] [JOB 41143] Generated table #291132: 942629 keys, 32276312 bytes, temperature: kWarm 2024/11/24-03:57:46.963107 139957584442944 EVENT_LOG_v1 {"time_micros": 1732391866963072, "cf_name": "default", "job": 41143, "event": "table_file_creation", "file_number": 291132, "file_size": 32276312, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4316484572, "largest_seqno": 4427728093, "table_properties": {"data_size": 26210000, "index_size": 1805376, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26393612, "raw_average_key_size": 28, "raw_value_size": 107824130, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 942629, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391863, "oldest_key_time": 0, "file_creation_time": 1732391863, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291132, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:57:47.005836 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291133.sst) /dev/shm/ToplingTemp/Topling-263632-8raB0N 2024/11/24-03:57:47.950379 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291133.sst): first pass time = 0.94's, 142.104'MB/sec 2024/11/24-03:57:49.286950 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291133.sst): old prealloc_size = 147639500, real_size = 32730912 2024/11/24-03:57:49.287544 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291133.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924148 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0307 GB avg-val = 117.23 avg-zval = 33.25 2024/11/24-03:57:49.288317 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291133.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:57:50.122487 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291133.sst): second pass time = 1.34's, 81.136'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.071'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3344.132'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.12's, 295.359'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 33158.685'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.136'MB/sec zip pipeline throughput = 81.136'MB/sec entries = 924148 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.23 avg-zval = 33.25 usrkeys = 924148 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.23 avg-zval = 33.25 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924148 bytes = 3234528 } seq expand size = 6469036 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.4466 value = 3.5256 dict = 1.00 all = 3.6450 } Zip/UnZip{ index = 0.0957 value = 0.2836 dict = 1.00 all = 0.2743 } ---------------------------- total value len = 2253.161443 GB avg = 0.199 KB (by entry num) total key len = 307.298448 GB avg = 0.027 KB total ukey len = 237.651503 GB avg = 0.015 KB total ukey num = 15.753499937 Billion total entry num = 11.294813885 Billion write speed all = 37.347404151 MB/sec (with seq num) write speed all = 36.029415987 MB/sec (without seq num) 2024/11/24-03:57:50.169599 139957584442944 [compaction_job.cc:1948] [default] [JOB 41143] Generated table #291133: 924148 keys, 36821938 bytes, temperature: kWarm 2024/11/24-03:57:50.169668 139957584442944 EVENT_LOG_v1 {"time_micros": 1732391870169632, "cf_name": "default", "job": 41143, "event": "table_file_creation", "file_number": 291133, "file_size": 36821938, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679865, "largest_seqno": 4427727128, "table_properties": {"data_size": 30730384, "index_size": 1769280, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25876144, "raw_average_key_size": 28, "raw_value_size": 108341612, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924148, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391866, "oldest_key_time": 0, "file_creation_time": 1732391866, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291133, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:57:50.206264 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291134.sst) /dev/shm/ToplingTemp/Topling-263632-0MbFtO 2024/11/24-03:57:51.008644 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291134.sst): first pass time = 0.80's, 167.284'MB/sec 2024/11/24-03:57:52.304885 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291134.sst): old prealloc_size = 147639500, real_size = 30243632 2024/11/24-03:57:52.305463 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291134.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930371 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0282 GB avg-val = 116.26 avg-zval = 30.34 2024/11/24-03:57:52.306216 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291134.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:57:53.120543 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291134.sst): second pass time = 1.30's, 83.498'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 41.972'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3208.765'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.338'MB/sec write SST data time = 0.13's, 271.723'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32238.195'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.498'MB/sec zip pipeline throughput = 83.498'MB/sec entries = 930371 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.26 avg-zval = 30.34 usrkeys = 930371 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.26 avg-zval = 30.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930371 bytes = 3140016 } seq expand size = 6512597 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0282 GB dict = 1.08 MB all = 0.0342 GB } UnZip/Zip{ index = 10.4433 value = 3.8318 dict = 1.00 all = 3.9206 } Zip/UnZip{ index = 0.0958 value = 0.2610 dict = 1.00 all = 0.2551 } ---------------------------- total value len = 2253.276123 GB avg = 0.199 KB (by entry num) total key len = 307.324499 GB avg = 0.027 KB total ukey len = 237.670110 GB avg = 0.015 KB total ukey num = 15.754430308 Billion total entry num = 11.295744256 Billion write speed all = 37.347823644 MB/sec (with seq num) write speed all = 36.029784553 MB/sec (without seq num) 2024/11/24-03:57:53.165319 139957584442944 [compaction_job.cc:1948] [default] [JOB 41143] Generated table #291134: 930371 keys, 34233714 bytes, temperature: kWarm 2024/11/24-03:57:53.165397 139957584442944 EVENT_LOG_v1 {"time_micros": 1732391873165354, "cf_name": "default", "job": 41143, "event": "table_file_creation", "file_number": 291134, "file_size": 34233714, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4359080171, "largest_seqno": 4427728111, "table_properties": {"data_size": 28229088, "index_size": 1781760, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26050388, "raw_average_key_size": 28, "raw_value_size": 108167410, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930371, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391870, "oldest_key_time": 0, "file_creation_time": 1732391870, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291134, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:57:53.206013 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291135.sst) /dev/shm/ToplingTemp/Topling-263632-zgaoRO 2024/11/24-03:57:54.175958 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291135.sst): first pass time = 0.97's, 138.383'MB/sec 2024/11/24-03:57:55.522215 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291135.sst): old prealloc_size = 147639500, real_size = 32715552 2024/11/24-03:57:55.522739 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291135.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924127 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0307 GB avg-val = 117.24 avg-zval = 33.25 2024/11/24-03:57:55.523513 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291135.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:57:56.356381 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291135.sst): second pass time = 1.34's, 80.554'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 44.125'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3620.627'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.12's, 295.697'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 34117.784'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.554'MB/sec zip pipeline throughput = 80.554'MB/sec entries = 924127 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.24 avg-zval = 33.25 usrkeys = 924127 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.24 avg-zval = 33.25 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924127 bytes = 3234464 } seq expand size = 6468889 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5114 value = 3.5261 dict = 1.00 all = 3.6466 } Zip/UnZip{ index = 0.0951 value = 0.2836 dict = 1.00 all = 0.2742 } ---------------------------- total value len = 2253.390934 GB avg = 0.199 KB (by entry num) total key len = 307.350374 GB avg = 0.027 KB total ukey len = 237.688593 GB avg = 0.015 KB total ukey num = 15.755354435 Billion total entry num = 11.296668383 Billion write speed all = 37.348112933 MB/sec (with seq num) write speed all = 36.030028221 MB/sec (without seq num) 2024/11/24-03:57:56.403856 139957584442944 [compaction_job.cc:1948] [default] [JOB 41143] Generated table #291135: 924127 keys, 36806418 bytes, temperature: kWarm 2024/11/24-03:57:56.403926 139957584442944 EVENT_LOG_v1 {"time_micros": 1732391876403890, "cf_name": "default", "job": 41143, "event": "table_file_creation", "file_number": 291135, "file_size": 36806418, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679880, "largest_seqno": 4427728371, "table_properties": {"data_size": 30726032, "index_size": 1758336, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25875556, "raw_average_key_size": 28, "raw_value_size": 108342226, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924127, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391873, "oldest_key_time": 0, "file_creation_time": 1732391873, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291135, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:57:56.445847 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291136.sst) /dev/shm/ToplingTemp/Topling-263632-qgXAmN 2024/11/24-03:57:57.391952 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291136.sst): first pass time = 0.95's, 141.870'MB/sec 2024/11/24-03:57:58.685630 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291136.sst): old prealloc_size = 147639500, real_size = 30565712 2024/11/24-03:57:58.686214 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291136.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928205 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0286 GB avg-val = 116.60 avg-zval = 30.77 2024/11/24-03:57:58.686970 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291136.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:57:59.508355 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291136.sst): second pass time = 1.29's, 83.735'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 41.996'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3162.180'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.333'MB/sec write SST data time = 0.13's, 275.473'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30963.583'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.735'MB/sec zip pipeline throughput = 83.735'MB/sec entries = 928205 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.60 avg-zval = 30.77 usrkeys = 928205 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.60 avg-zval = 30.77 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928205 bytes = 3132704 } seq expand size = 6497435 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0286 GB dict = 1.08 MB all = 0.0346 GB } UnZip/Zip{ index = 10.4464 value = 3.7900 dict = 1.00 all = 3.8847 } Zip/UnZip{ index = 0.0957 value = 0.2639 dict = 1.00 all = 0.2574 } ---------------------------- total value len = 2253.505660 GB avg = 0.199 KB (by entry num) total key len = 307.376364 GB avg = 0.027 KB total ukey len = 237.707157 GB avg = 0.015 KB total ukey num = 15.756282640 Billion total entry num = 11.297596588 Billion write speed all = 37.348448301 MB/sec (with seq num) write speed all = 36.030315882 MB/sec (without seq num) 2024/11/24-03:57:59.551677 139957584442944 [compaction_job.cc:1948] [default] [JOB 41143] Generated table #291136: 928205 keys, 34550176 bytes, temperature: kWarm 2024/11/24-03:57:59.551749 139957584442944 EVENT_LOG_v1 {"time_micros": 1732391879551712, "cf_name": "default", "job": 41143, "event": "table_file_creation", "file_number": 291136, "file_size": 34550176, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4353802331, "largest_seqno": 4427728121, "table_properties": {"data_size": 28556416, "index_size": 1777088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25989740, "raw_average_key_size": 28, "raw_value_size": 108228082, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928205, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391876, "oldest_key_time": 0, "file_creation_time": 1732391876, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291136, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:57:59.601887 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291137.sst) /dev/shm/ToplingTemp/Topling-263632-jQ8igN 2024/11/24-03:57:59.705486 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291137.sst): first pass time = 0.10's, 232.214'MB/sec 2024/11/24-03:57:59.926020 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291137.sst): old prealloc_size = 147639500, real_size = 6758736 2024/11/24-03:57:59.926171 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291137.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 168378 prefix = 4 raw-key = 0.0027 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0193 GB zip-val = 0.0064 GB avg-val = 114.83 avg-zval = 37.98 2024/11/24-03:57:59.926360 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291137.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0004 GB, waited 0.000 sec, Key+Value = 0.024 GB 2024/11/24-03:58:00.072039 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00/291137.sst): second pass time = 0.22's, 87.948'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 51.439'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2489.434'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.343'MB/sec write SST data time = 0.02's, 324.388'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.203'MB, throughput = 6505.347'MB/sec zip my value time = 0.22's, unzip length = 0.019'GB zip my value throughput = 87.948'MB/sec zip pipeline throughput = 87.948'MB/sec entries = 168378 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.83 avg-zval = 37.98 usrkeys = 168378 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.83 avg-zval = 37.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 168378 bytes = 589344 } seq expand size = 1178646 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0034 GB value = 0.0193 GB dict = 0.20 MB all = 0.0240 GB } __ZipSize{ index = 0.0003 GB value = 0.0064 GB dict = 0.20 MB all = 0.0075 GB } UnZip/Zip{ index = 10.4713 value = 3.0235 dict = 1.00 all = 3.2018 } Zip/UnZip{ index = 0.0955 value = 0.3307 dict = 1.00 all = 0.3123 } ---------------------------- total value len = 2253.526173 GB avg = 0.199 KB (by entry num) total key len = 307.381078 GB avg = 0.027 KB total ukey len = 237.710524 GB avg = 0.015 KB total ukey num = 15.756451018 Billion total entry num = 11.297764966 Billion write speed all = 37.348509189 MB/sec (with seq num) write speed all = 36.030367961 MB/sec (without seq num) 2024/11/24-03:58:00.088137 139957584442944 [compaction_job.cc:1948] [default] [JOB 41143] Generated table #291137: 168378 keys, 7511209 bytes, temperature: kWarm 2024/11/24-03:58:00.088195 139957584442944 EVENT_LOG_v1 {"time_micros": 1732391880088165, "cf_name": "default", "job": 41143, "event": "table_file_creation", "file_number": 291137, "file_size": 7511209, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679891, "largest_seqno": 4427728282, "table_properties": {"data_size": 6394880, "index_size": 321600, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4714584, "raw_average_key_size": 28, "raw_value_size": 19334636, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 168378, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391879, "oldest_key_time": 0, "file_creation_time": 1732391879, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291137, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:58:00.106228 139957584442944 [compaction_job.cc:703] job-41143: subcompact[0], size: 0.684476 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41143/att-00 [291118,291119,291120,291121,291122,291123,291124,291125,291126,291127,291128,291129,291130,291131,291132,291133,291134,291135,291136,291137] 2024/11/24-03:58:00.220187 139957584442944 (Original Log Time 2024/11/24-03:56:58.189166) EVENT_LOG_v1 {"time_micros": 1732391818189122, "job": 41143, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [291058], "files_L3": [291114, 291115, 291014, 290973, 290974, 290975, 290976, 291015, 291016, 291017, 291018, 291019, 291020, 291021, 291022, 291023, 291024, 291025, 291026, 291027, 291089], "score": 0.973416, "input_data_size": 706034068, "oldest_snapshot_seqno": 4428111012} 2024/11/24-03:58:00.220190 139957584442944 (Original Log Time 2024/11/24-03:58:00.218630) [compaction_job.cc:2022] [default] [JOB 41143] Compacted 1@2 + 21@3 files to L3 => 684475793 bytes 2024/11/24-03:58:00.220191 139957584442944 (Original Log Time 2024/11/24-03:58:00.220112) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 20 0 0 0] max score 0.03, MB/sec: 11.4 rd, 11.1 wr, level 3, files in(1, 21) out(20 +0 blob) MB in(27.1, 646.2 +0.0 blob) out(652.8 +0.0 blob), read-write-amplify(48.9) write-amplify(24.1) OK, records in: 17897876, records dropped: 108021 output_compression: Snappy 2024/11/24-03:58:00.220195 139957584442944 (Original Log Time 2024/11/24-03:58:00.220160) EVENT_LOG_v1 {"time_micros": 1732391880220136, "cf": "default", "job": 41143, "event": "compaction_finished", "compaction_time_micros": 61917077, "compaction_time_cpu_micros": 26508652, "output_level": 3, "num_output_files": 20, "total_output_size": 684475793, "num_input_records": 17897876, "num_output_records": 17789855, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 469157558, "file_range_sync_nanos": 0, "file_fsync_nanos": 12539, "file_prepare_write_nanos": 57783, "lsm_state": [0, 0, 0, 20, 0, 0, 0]} 2024/11/24-03:58:00.336052 139957584442944 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732391818,"executesMs":62154,"compactionJobId":41143,"attempt":0,"compactionInputRawBytes":2589525992,"compactionInputZipBytes":706034068,"compactionOutputRawBytes":0,"compactionOutputZipBytes":684475793,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-6748e3c230a4b5ea2ff27e74d60c2d31-c9552f4a5cedfbe5-00"}}, response = 2024/11/24-03:58:00.336154 139957584442944 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41143/att-00: olev 3, work 62.154 s, result 7.766 ms, install 1.579 ms, input{raw 2.412 GiB zip 673.327 MiB}