2024/11/24-06:26:38.623663 139957399803456 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00: fork to child time = 0.008417 sec 2024/11/24-06:26:38.624688 139957399803456 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00 2024/11/24-06:26:38.632832 139957399803456 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00 2024/11/24-06:26:38.633746 139957399803456 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-06:26:38.634007 139957399803456 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-06:26:38.691075 139957399803456 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-42806/att-00/MANIFEST-206347 2024/11/24-06:26:38.719166 139957399803456 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-42806/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 294906, last_sequence is 4478453786, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 294762 2024/11/24-06:26:38.719174 139957399803456 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-06:26:38.719333 139957399803456 [version_set.cc:5721] Creating manifest 294906 2024/11/24-06:26:38.752963 139957399803456 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-06:26:38.774235 139957399803456 [compaction_job.cc:2437] [default] [JOB 42806] Compacting 1@2 + 28@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/24-06:26:38.774255 139957399803456 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 32084 Base level 2, inputs: [294899(27MB)], [294869(35MB) 294870(31MB) 294871(4603KB) 294825(30MB) 294826(35MB) 294827(32MB) 294828(32MB) 294829(33MB) 294830(32MB) 294831(35MB) 294832(33MB) 294833(34MB) 294834(33MB) 294835(33MB) 294836(35MB) 294837(33MB) 294838(35MB) 294839(33MB) 294840(32MB) 294841(34MB) 294842(33MB) 294843(35MB) 294844(33MB) 294845(33MB) 294846(33MB) 294847(33MB) 294848(10MB) 294797(402KB)] 2024/11/24-06:26:38.781479 139957399803456 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-06:26:38.781508 139957399803456 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-06:26:38.781528 139957399803456 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-06:26:38.781534 139957399803456 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-06:26:38.781544 139957399803456 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-06:26:39.387085 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294907.sst) /dev/shm/ToplingTemp/Topling-288697-LMAoY5 2024/11/24-06:26:39.958230 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294907.sst): first pass time = 0.57's, 235.014'MB/sec 2024/11/24-06:26:42.588778 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294907.sst): old prealloc_size = 147639500, real_size = 33468512 2024/11/24-06:26:42.589491 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294907.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927963 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0315 GB avg-val = 116.64 avg-zval = 33.90 2024/11/24-06:26:42.590358 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294907.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:26:44.353484 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294907.sst): second pass time = 2.63's, 41.160'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.76's, 21.292'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2554.042'MB/sec (index lex order gen) rebuild zvType time = 1.42's, 0.164'MB/sec write SST data time = 0.35's, 108.137'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35407.150'MB/sec zip my value time = 2.63's, unzip length = 0.108'GB zip my value throughput = 41.160'MB/sec zip pipeline throughput = 123.728'MB/sec entries = 927963 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.64 avg-zval = 33.90 usrkeys = 927963 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.64 avg-zval = 33.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927963 bytes = 3247888 } seq expand size = 6495741 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.4463 value = 3.4404 dict = 1.00 all = 3.5721 } Zip/UnZip{ index = 0.0957 value = 0.2907 dict = 1.00 all = 0.2799 } ---------------------------- total value len = 2251.440304 GB avg = 0.200 KB (by entry num) total key len = 306.908945 GB avg = 0.027 KB total ukey len = 237.373287 GB avg = 0.015 KB total ukey num = 15.739589126 Billion total entry num = 11.280903074 Billion write speed all = 33.014306851 MB/sec (with seq num) write speed all = 31.849708403 MB/sec (without seq num) 2024/11/24-06:26:44.410105 139957399803456 [compaction_job.cc:1948] [default] [JOB 42806] Generated table #294907: 927963 keys, 37573532 bytes, temperature: kWarm 2024/11/24-06:26:44.410250 139957399803456 EVENT_LOG_v1 {"time_micros": 1732400804410178, "cf_name": "default", "job": 42806, "event": "table_file_creation", "file_number": 294907, "file_size": 37573532, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4478439935, "table_properties": {"data_size": 31459728, "index_size": 1776640, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25982964, "raw_average_key_size": 28, "raw_value_size": 108234818, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927963, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400799, "oldest_key_time": 0, "file_creation_time": 1732400799, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294907, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:26:44.477764 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294908.sst) /dev/shm/ToplingTemp/Topling-288697-TV3n86 2024/11/24-06:26:45.589404 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294908.sst): first pass time = 1.11's, 120.745'MB/sec 2024/11/24-06:26:48.593087 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294908.sst): old prealloc_size = 147639500, real_size = 28941840 2024/11/24-06:26:48.593814 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294908.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 939603 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1079 GB zip-val = 0.0269 GB avg-val = 114.85 avg-zval = 28.64 2024/11/24-06:26:48.594745 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294908.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:26:49.837802 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294908.sst): second pass time = 3.00's, 35.936'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.24's, 26.574'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2566.357'MB/sec (index lex order gen) rebuild zvType time = 1.09's, 0.215'MB/sec write SST data time = 0.15's, 216.813'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 38058.163'MB/sec zip my value time = 3.00's, unzip length = 0.108'GB zip my value throughput = 35.936'MB/sec zip pipeline throughput = 122.960'MB/sec entries = 939603 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.85 avg-zval = 28.64 usrkeys = 939603 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.85 avg-zval = 28.64 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 939603 bytes = 3288624 } seq expand size = 6577221 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0188 GB value = 0.1079 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0269 GB dict = 1.08 MB all = 0.0331 GB } UnZip/Zip{ index = 10.4449 value = 4.0104 dict = 1.00 all = 4.0578 } Zip/UnZip{ index = 0.0957 value = 0.2494 dict = 1.00 all = 0.2464 } ---------------------------- total value len = 2251.554790 GB avg = 0.200 KB (by entry num) total key len = 306.935254 GB avg = 0.027 KB total ukey len = 237.392079 GB avg = 0.015 KB total ukey num = 15.740528729 Billion total entry num = 11.281842677 Billion write speed all = 33.013787267 MB/sec (with seq num) write speed all = 31.849174241 MB/sec (without seq num) 2024/11/24-06:26:49.930480 139957399803456 [compaction_job.cc:1948] [default] [JOB 42806] Generated table #294908: 939603 keys, 33076614 bytes, temperature: kWarm 2024/11/24-06:26:49.930526 139957399803456 EVENT_LOG_v1 {"time_micros": 1732400809930500, "cf_name": "default", "job": 42806, "event": "table_file_creation", "file_number": 294908, "file_size": 33076614, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680463, "largest_seqno": 4478441077, "table_properties": {"data_size": 26907584, "index_size": 1799168, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26308884, "raw_average_key_size": 28, "raw_value_size": 107908910, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 939603, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400804, "oldest_key_time": 0, "file_creation_time": 1732400804, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294908, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:26:49.987547 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294909.sst) /dev/shm/ToplingTemp/Topling-288697-CEGpA6 2024/11/24-06:26:50.960634 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294909.sst): first pass time = 0.97's, 137.935'MB/sec 2024/11/24-06:26:52.613354 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294909.sst): old prealloc_size = 147639500, real_size = 27851792 2024/11/24-06:26:52.613925 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294909.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 945196 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1078 GB zip-val = 0.0258 GB avg-val = 114.00 avg-zval = 27.30 2024/11/24-06:26:52.614698 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294909.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:26:53.465118 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294909.sst): second pass time = 1.65's, 65.224'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 37.433'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3334.301'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.329'MB/sec write SST data time = 0.13's, 240.091'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.073'MB, throughput = 26752.000'MB/sec zip my value time = 1.65's, unzip length = 0.108'GB zip my value throughput = 65.224'MB/sec zip pipeline throughput = 127.253'MB/sec entries = 945196 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.00 avg-zval = 27.30 usrkeys = 945196 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.00 avg-zval = 27.30 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 945196 bytes = 3190048 } seq expand size = 6616372 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0258 GB dict = 1.07 MB all = 0.0319 GB } UnZip/Zip{ index = 10.4454 value = 4.1756 dict = 1.00 all = 4.2097 } Zip/UnZip{ index = 0.0957 value = 0.2395 dict = 1.00 all = 0.2375 } ---------------------------- total value len = 2251.669159 GB avg = 0.200 KB (by entry num) total key len = 306.961720 GB avg = 0.027 KB total ukey len = 237.410983 GB avg = 0.015 KB total ukey num = 15.741473925 Billion total entry num = 11.282787873 Billion write speed all = 33.014059297 MB/sec (with seq num) write speed all = 31.849403212 MB/sec (without seq num) 2024/11/24-06:26:53.510659 139957399803456 [compaction_job.cc:1948] [default] [JOB 42806] Generated table #294909: 945196 keys, 31882992 bytes, temperature: kWarm 2024/11/24-06:26:53.510729 139957399803456 EVENT_LOG_v1 {"time_micros": 1732400813510689, "cf_name": "default", "job": 42806, "event": "table_file_creation", "file_number": 294909, "file_size": 31882992, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4343763754, "largest_seqno": 4446488124, "table_properties": {"data_size": 25805504, "index_size": 1809792, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26465488, "raw_average_key_size": 28, "raw_value_size": 107752344, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 945196, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400809, "oldest_key_time": 0, "file_creation_time": 1732400809, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294909, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:26:53.548426 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294910.sst) /dev/shm/ToplingTemp/Topling-288697-kR1vl5 2024/11/24-06:26:54.429013 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294910.sst): first pass time = 0.88's, 152.425'MB/sec 2024/11/24-06:26:56.063252 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294910.sst): old prealloc_size = 147639500, real_size = 33076256 2024/11/24-06:26:56.063853 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294910.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927164 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 116.76 avg-zval = 33.52 2024/11/24-06:26:56.064657 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294910.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:26:56.932842 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294910.sst): second pass time = 1.63's, 66.277'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.752'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3063.445'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.13's, 280.927'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 18712.996'MB/sec zip my value time = 1.63's, unzip length = 0.108'GB zip my value throughput = 66.277'MB/sec zip pipeline throughput = 131.292'MB/sec entries = 927164 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.76 avg-zval = 33.52 usrkeys = 927164 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.76 avg-zval = 33.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927164 bytes = 3245088 } seq expand size = 6490148 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5130 value = 3.4831 dict = 1.00 all = 3.6104 } Zip/UnZip{ index = 0.0951 value = 0.2871 dict = 1.00 all = 0.2770 } ---------------------------- total value len = 2251.783906 GB avg = 0.200 KB (by entry num) total key len = 306.987680 GB avg = 0.027 KB total ukey len = 237.429526 GB avg = 0.015 KB total ukey num = 15.742401089 Billion total entry num = 11.283715037 Billion write speed all = 33.014397652 MB/sec (with seq num) write speed all = 31.849697975 MB/sec (without seq num) 2024/11/24-06:26:56.980888 139957399803456 [compaction_job.cc:1948] [default] [JOB 42806] Generated table #294910: 927164 keys, 37175696 bytes, temperature: kWarm 2024/11/24-06:26:56.980957 139957399803456 EVENT_LOG_v1 {"time_micros": 1732400816980922, "cf_name": "default", "job": 42806, "event": "table_file_creation", "file_number": 294910, "file_size": 37175696, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4478438700, "table_properties": {"data_size": 31080464, "index_size": 1763840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25960592, "raw_average_key_size": 28, "raw_value_size": 108257200, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927164, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400813, "oldest_key_time": 0, "file_creation_time": 1732400813, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294910, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:26:57.019045 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294911.sst) /dev/shm/ToplingTemp/Topling-288697-chnQu5 2024/11/24-06:26:57.889128 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294911.sst): first pass time = 0.87's, 154.265'MB/sec 2024/11/24-06:26:59.485039 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294911.sst): old prealloc_size = 147639500, real_size = 30899888 2024/11/24-06:26:59.485630 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294911.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925690 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0289 GB avg-val = 116.99 avg-zval = 31.20 2024/11/24-06:26:59.486409 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294911.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:27:00.315502 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294911.sst): second pass time = 1.60's, 67.893'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 41.998'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3140.857'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.14's, 247.309'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 24796.595'MB/sec zip my value time = 1.60's, unzip length = 0.108'GB zip my value throughput = 67.893'MB/sec zip pipeline throughput = 120.532'MB/sec entries = 925690 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.99 avg-zval = 31.20 usrkeys = 925690 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.99 avg-zval = 31.20 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925690 bytes = 3124224 } seq expand size = 6479830 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0349 GB } UnZip/Zip{ index = 10.3766 value = 3.7494 dict = 1.00 all = 3.8483 } Zip/UnZip{ index = 0.0964 value = 0.2667 dict = 1.00 all = 0.2599 } ---------------------------- total value len = 2251.898684 GB avg = 0.200 KB (by entry num) total key len = 307.013600 GB avg = 0.027 KB total ukey len = 237.448040 GB avg = 0.015 KB total ukey num = 15.743326779 Billion total entry num = 11.284640727 Billion write speed all = 33.014772077 MB/sec (with seq num) write speed all = 31.850027685 MB/sec (without seq num) 2024/11/24-06:27:00.359302 139957399803456 [compaction_job.cc:1948] [default] [JOB 42806] Generated table #294911: 925690 keys, 34877274 bytes, temperature: kWarm 2024/11/24-06:27:00.359362 139957399803456 EVENT_LOG_v1 {"time_micros": 1732400820359329, "cf_name": "default", "job": 42806, "event": "table_file_creation", "file_number": 294911, "file_size": 34877274, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4348627826, "largest_seqno": 4478441091, "table_properties": {"data_size": 28884064, "index_size": 1784192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25919320, "raw_average_key_size": 28, "raw_value_size": 108298408, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925690, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400817, "oldest_key_time": 0, "file_creation_time": 1732400817, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294911, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:27:00.396468 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294912.sst) /dev/shm/ToplingTemp/Topling-288697-c1jzd5 2024/11/24-06:27:01.344657 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294912.sst): first pass time = 0.95's, 141.558'MB/sec 2024/11/24-06:27:02.608020 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294912.sst): old prealloc_size = 147639500, real_size = 27924496 2024/11/24-06:27:02.608637 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294912.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 945196 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1078 GB zip-val = 0.0259 GB avg-val = 114.00 avg-zval = 27.38 2024/11/24-06:27:02.609438 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294912.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:27:03.455760 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294912.sst): second pass time = 1.26's, 85.381'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 37.695'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3057.049'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.331'MB/sec write SST data time = 0.13's, 240.763'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.073'MB, throughput = 34556.353'MB/sec zip my value time = 1.26's, unzip length = 0.108'GB zip my value throughput = 85.381'MB/sec zip pipeline throughput = 85.381'MB/sec entries = 945196 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.00 avg-zval = 27.38 usrkeys = 945196 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.00 avg-zval = 27.38 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 945196 bytes = 3190048 } seq expand size = 6616372 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0259 GB dict = 1.07 MB all = 0.0320 GB } UnZip/Zip{ index = 10.4439 value = 4.1639 dict = 1.00 all = 4.2001 } Zip/UnZip{ index = 0.0957 value = 0.2402 dict = 1.00 all = 0.2381 } ---------------------------- total value len = 2252.013053 GB avg = 0.200 KB (by entry num) total key len = 307.040065 GB avg = 0.027 KB total ukey len = 237.466944 GB avg = 0.015 KB total ukey num = 15.744271975 Billion total entry num = 11.285585923 Billion write speed all = 33.015251487 MB/sec (with seq num) write speed all = 31.850456728 MB/sec (without seq num) 2024/11/24-06:27:03.496314 139957399803456 [compaction_job.cc:1948] [default] [JOB 42806] Generated table #294912: 945196 keys, 31955696 bytes, temperature: kWarm 2024/11/24-06:27:03.496393 139957399803456 EVENT_LOG_v1 {"time_micros": 1732400823496350, "cf_name": "default", "job": 42806, "event": "table_file_creation", "file_number": 294912, "file_size": 31955696, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4373430017, "largest_seqno": 4477282890, "table_properties": {"data_size": 25877952, "index_size": 1810048, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26465488, "raw_average_key_size": 28, "raw_value_size": 107752344, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 945196, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400820, "oldest_key_time": 0, "file_creation_time": 1732400820, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294912, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:27:03.531599 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294913.sst) /dev/shm/ToplingTemp/Topling-288697-HSgii5 2024/11/24-06:27:04.528393 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294913.sst): first pass time = 1.00's, 134.655'MB/sec 2024/11/24-06:27:05.841602 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294913.sst): old prealloc_size = 147639500, real_size = 31474896 2024/11/24-06:27:05.842200 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294913.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919596 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 117.95 avg-zval = 32.06 2024/11/24-06:27:05.843619 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294913.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:27:06.655430 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294913.sst): second pass time = 1.31's, 82.707'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.690'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3012.375'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.336'MB/sec write SST data time = 0.13's, 276.941'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 36846.198'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.707'MB/sec zip pipeline throughput = 82.707'MB/sec entries = 919596 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.95 avg-zval = 32.06 usrkeys = 919596 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.95 avg-zval = 32.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919596 bytes = 3218608 } seq expand size = 6437172 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0356 GB } UnZip/Zip{ index = 10.4462 value = 3.6789 dict = 1.00 all = 3.7750 } Zip/UnZip{ index = 0.0957 value = 0.2718 dict = 1.00 all = 0.2649 } ---------------------------- total value len = 2252.127959 GB avg = 0.200 KB (by entry num) total key len = 307.065814 GB avg = 0.027 KB total ukey len = 237.485335 GB avg = 0.015 KB total ukey num = 15.745191571 Billion total entry num = 11.286505519 Billion write speed all = 33.015703238 MB/sec (with seq num) write speed all = 31.850861652 MB/sec (without seq num) 2024/11/24-06:27:06.699564 139957399803456 [compaction_job.cc:1948] [default] [JOB 42806] Generated table #294913: 919596 keys, 35554338 bytes, temperature: kWarm 2024/11/24-06:27:06.699629 139957399803456 EVENT_LOG_v1 {"time_micros": 1732400826699594, "cf_name": "default", "job": 42806, "event": "table_file_creation", "file_number": 294913, "file_size": 35554338, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679865, "largest_seqno": 4478438533, "table_properties": {"data_size": 29484160, "index_size": 1760640, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25748688, "raw_average_key_size": 28, "raw_value_size": 108469096, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 919596, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400823, "oldest_key_time": 0, "file_creation_time": 1732400823, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294913, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:27:06.738001 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294914.sst) /dev/shm/ToplingTemp/Topling-288697-Dj8cp7 2024/11/24-06:27:07.728538 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294914.sst): first pass time = 0.99's, 135.506'MB/sec 2024/11/24-06:27:09.043177 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294914.sst): old prealloc_size = 147639500, real_size = 30986208 2024/11/24-06:27:09.043757 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294914.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924297 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0290 GB avg-val = 117.21 avg-zval = 31.36 2024/11/24-06:27:09.044519 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294914.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:27:09.860958 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294914.sst): second pass time = 1.31's, 82.514'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.752'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3179.849'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 271.045'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 32668.861'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.514'MB/sec zip pipeline throughput = 82.514'MB/sec entries = 924297 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.21 avg-zval = 31.36 usrkeys = 924297 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.21 avg-zval = 31.36 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924297 bytes = 3119520 } seq expand size = 6470079 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0290 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.4419 value = 3.7378 dict = 1.00 all = 3.8391 } Zip/UnZip{ index = 0.0958 value = 0.2675 dict = 1.00 all = 0.2605 } ---------------------------- total value len = 2252.242767 GB avg = 0.200 KB (by entry num) total key len = 307.091694 GB avg = 0.027 KB total ukey len = 237.503821 GB avg = 0.015 KB total ukey num = 15.746115868 Billion total entry num = 11.287429816 Billion write speed all = 33.016152876 MB/sec (with seq num) write speed all = 31.851264069 MB/sec (without seq num) 2024/11/24-06:27:09.904356 139957399803456 [compaction_job.cc:1948] [default] [JOB 42806] Generated table #294914: 924297 keys, 34960568 bytes, temperature: kWarm 2024/11/24-06:27:09.904431 139957399803456 EVENT_LOG_v1 {"time_micros": 1732400829904396, "cf_name": "default", "job": 42806, "event": "table_file_creation", "file_number": 294914, "file_size": 34960568, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4379944538, "largest_seqno": 4478441109, "table_properties": {"data_size": 28984592, "index_size": 1770368, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25880316, "raw_average_key_size": 28, "raw_value_size": 108337474, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924297, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400826, "oldest_key_time": 0, "file_creation_time": 1732400826, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294914, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:27:09.951549 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294915.sst) /dev/shm/ToplingTemp/Topling-288697-Ms9H06 2024/11/24-06:27:10.782279 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294915.sst): first pass time = 0.83's, 161.574'MB/sec 2024/11/24-06:27:12.130667 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294915.sst): old prealloc_size = 147639500, real_size = 32758704 2024/11/24-06:27:12.131243 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294915.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932535 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0307 GB avg-val = 115.93 avg-zval = 32.96 2024/11/24-06:27:12.132054 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294915.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:27:12.995900 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294915.sst): second pass time = 1.35's, 80.255'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.618'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3185.176'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 283.235'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32667.051'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.255'MB/sec zip pipeline throughput = 80.255'MB/sec entries = 932535 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.93 avg-zval = 32.96 usrkeys = 932535 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.93 avg-zval = 32.96 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932535 bytes = 3263888 } seq expand size = 6527745 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4447 value = 3.5168 dict = 1.00 all = 3.6400 } Zip/UnZip{ index = 0.0957 value = 0.2843 dict = 1.00 all = 0.2747 } ---------------------------- total value len = 2252.357401 GB avg = 0.200 KB (by entry num) total key len = 307.117805 GB avg = 0.027 KB total ukey len = 237.522472 GB avg = 0.015 KB total ukey num = 15.747048403 Billion total entry num = 11.288362351 Billion write speed all = 33.016633287 MB/sec (with seq num) write speed all = 31.851695352 MB/sec (without seq num) 2024/11/24-06:27:13.040545 139957399803456 [compaction_job.cc:1948] [default] [JOB 42806] Generated table #294915: 932535 keys, 36873030 bytes, temperature: kWarm 2024/11/24-06:27:13.040612 139957399803456 EVENT_LOG_v1 {"time_micros": 1732400833040578, "cf_name": "default", "job": 42806, "event": "table_file_creation", "file_number": 294915, "file_size": 36873030, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679880, "largest_seqno": 4478439778, "table_properties": {"data_size": 30739744, "index_size": 1785664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26110980, "raw_average_key_size": 28, "raw_value_size": 108106810, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932535, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400829, "oldest_key_time": 0, "file_creation_time": 1732400829, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294915, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:27:13.085907 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294916.sst) /dev/shm/ToplingTemp/Topling-288697-xqrmS8 2024/11/24-06:27:14.081634 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294916.sst): first pass time = 1.00's, 134.799'MB/sec 2024/11/24-06:27:15.378307 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294916.sst): old prealloc_size = 147639500, real_size = 31292624 2024/11/24-06:27:15.378849 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294916.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0293 GB avg-val = 118.00 avg-zval = 31.87 2024/11/24-06:27:15.379587 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294916.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:27:16.189315 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294916.sst): second pass time = 1.30's, 83.708'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.471'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3386.370'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.337'MB/sec write SST data time = 0.13's, 276.089'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 36280.679'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.708'MB/sec zip pipeline throughput = 83.708'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.87 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4447 value = 3.7020 dict = 1.00 all = 3.8071 } Zip/UnZip{ index = 0.0957 value = 0.2701 dict = 1.00 all = 0.2627 } ---------------------------- total value len = 2252.472314 GB avg = 0.200 KB (by entry num) total key len = 307.143546 GB avg = 0.027 KB total ukey len = 237.540858 GB avg = 0.015 KB total ukey num = 15.747967703 Billion total entry num = 11.289281651 Billion write speed all = 33.017087560 MB/sec (with seq num) write speed all = 31.852102746 MB/sec (without seq num) 2024/11/24-06:27:16.232485 139957399803456 [compaction_job.cc:1948] [default] [JOB 42806] Generated table #294916: 919300 keys, 35254450 bytes, temperature: kWarm 2024/11/24-06:27:16.232552 139957399803456 EVENT_LOG_v1 {"time_micros": 1732400836232517, "cf_name": "default", "job": 42806, "event": "table_file_creation", "file_number": 294916, "file_size": 35254450, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4318027049, "largest_seqno": 4429914598, "table_properties": {"data_size": 29302272, "index_size": 1760320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400833, "oldest_key_time": 0, "file_creation_time": 1732400833, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294916, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:27:16.268535 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294917.sst) /dev/shm/ToplingTemp/Topling-288697-VB6554 2024/11/24-06:27:17.252935 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294917.sst): first pass time = 0.98's, 136.350'MB/sec 2024/11/24-06:27:18.566221 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294917.sst): old prealloc_size = 147639500, real_size = 30304608 2024/11/24-06:27:18.566774 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294917.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933325 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0283 GB avg-val = 115.81 avg-zval = 30.32 2024/11/24-06:27:18.567561 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294917.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:27:19.401271 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294917.sst): second pass time = 1.31's, 82.352'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 41.223'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3353.902'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.13's, 265.082'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33909.863'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.352'MB/sec zip pipeline throughput = 82.352'MB/sec entries = 933325 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.81 avg-zval = 30.32 usrkeys = 933325 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.81 avg-zval = 30.32 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933325 bytes = 3266656 } seq expand size = 6533275 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0283 GB dict = 1.08 MB all = 0.0344 GB } UnZip/Zip{ index = 10.5123 value = 3.8199 dict = 1.00 all = 3.8992 } Zip/UnZip{ index = 0.0951 value = 0.2618 dict = 1.00 all = 0.2565 } ---------------------------- total value len = 2252.586932 GB avg = 0.200 KB (by entry num) total key len = 307.169679 GB avg = 0.027 KB total ukey len = 237.559525 GB avg = 0.015 KB total ukey num = 15.748901028 Billion total entry num = 11.290214976 Billion write speed all = 33.017535167 MB/sec (with seq num) write speed all = 31.852502309 MB/sec (without seq num) 2024/11/24-06:27:19.443122 139957399803456 [compaction_job.cc:1948] [default] [JOB 42806] Generated table #294917: 933325 keys, 34422022 bytes, temperature: kWarm 2024/11/24-06:27:19.443191 139957399803456 EVENT_LOG_v1 {"time_micros": 1732400839443156, "cf_name": "default", "job": 42806, "event": "table_file_creation", "file_number": 294917, "file_size": 34422022, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679891, "largest_seqno": 4478441122, "table_properties": {"data_size": 28295440, "index_size": 1775680, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26133100, "raw_average_key_size": 28, "raw_value_size": 108084710, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933325, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400836, "oldest_key_time": 0, "file_creation_time": 1732400836, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294917, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:27:19.479356 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294918.sst) /dev/shm/ToplingTemp/Topling-288697-HC9065 2024/11/24-06:27:20.472633 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294918.sst): first pass time = 0.99's, 135.132'MB/sec 2024/11/24-06:27:21.779963 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294918.sst): old prealloc_size = 147639500, real_size = 31323584 2024/11/24-06:27:21.780525 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294918.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0293 GB avg-val = 118.00 avg-zval = 31.91 2024/11/24-06:27:21.781269 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294918.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:27:22.594486 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294918.sst): second pass time = 1.31's, 83.026'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.322'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3380.083'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 276.392'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 27110.339'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 83.026'MB/sec zip pipeline throughput = 83.026'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.91 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4455 value = 3.6981 dict = 1.00 all = 3.8038 } Zip/UnZip{ index = 0.0957 value = 0.2704 dict = 1.00 all = 0.2629 } ---------------------------- total value len = 2252.701844 GB avg = 0.200 KB (by entry num) total key len = 307.195419 GB avg = 0.027 KB total ukey len = 237.577911 GB avg = 0.015 KB total ukey num = 15.749820328 Billion total entry num = 11.291134276 Billion write speed all = 33.017989453 MB/sec (with seq num) write speed all = 31.852909721 MB/sec (without seq num) 2024/11/24-06:27:22.638658 139957399803456 [compaction_job.cc:1948] [default] [JOB 42806] Generated table #294918: 919300 keys, 35285426 bytes, temperature: kWarm 2024/11/24-06:27:22.638730 139957399803456 EVENT_LOG_v1 {"time_micros": 1732400842638693, "cf_name": "default", "job": 42806, "event": "table_file_creation", "file_number": 294918, "file_size": 35285426, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4269109995, "largest_seqno": 4377222457, "table_properties": {"data_size": 29333360, "index_size": 1760192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400839, "oldest_key_time": 0, "file_creation_time": 1732400839, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294918, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:27:22.678469 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294919.sst) /dev/shm/ToplingTemp/Topling-288697-llxgA8 2024/11/24-06:27:23.672632 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294919.sst): first pass time = 0.99's, 135.011'MB/sec 2024/11/24-06:27:24.983818 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294919.sst): old prealloc_size = 147639500, real_size = 31231760 2024/11/24-06:27:24.984387 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294919.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921951 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0292 GB avg-val = 117.58 avg-zval = 31.71 2024/11/24-06:27:24.985128 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294919.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:27:25.799139 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294919.sst): second pass time = 1.31's, 82.726'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.175'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3240.258'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 279.316'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34668.501'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.726'MB/sec zip pipeline throughput = 82.726'MB/sec entries = 921951 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.58 avg-zval = 31.71 usrkeys = 921951 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.58 avg-zval = 31.71 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921951 bytes = 3111600 } seq expand size = 6453657 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0352 GB } UnZip/Zip{ index = 10.4452 value = 3.7079 dict = 1.00 all = 3.8130 } Zip/UnZip{ index = 0.0957 value = 0.2697 dict = 1.00 all = 0.2623 } ---------------------------- total value len = 2252.816701 GB avg = 0.200 KB (by entry num) total key len = 307.221234 GB avg = 0.027 KB total ukey len = 237.596350 GB avg = 0.015 KB total ukey num = 15.750742279 Billion total entry num = 11.292056227 Billion write speed all = 33.018439067 MB/sec (with seq num) write speed all = 31.853312362 MB/sec (without seq num) 2024/11/24-06:27:25.843123 139957399803456 [compaction_job.cc:1948] [default] [JOB 42806] Generated table #294919: 921951 keys, 35200172 bytes, temperature: kWarm 2024/11/24-06:27:25.843196 139957399803456 EVENT_LOG_v1 {"time_micros": 1732400845843158, "cf_name": "default", "job": 42806, "event": "table_file_creation", "file_number": 294919, "file_size": 35200172, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4377222458, "largest_seqno": 4478441140, "table_properties": {"data_size": 29235776, "index_size": 1765312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25814628, "raw_average_key_size": 28, "raw_value_size": 108403198, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921951, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400842, "oldest_key_time": 0, "file_creation_time": 1732400842, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294919, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:27:25.879881 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294920.sst) /dev/shm/ToplingTemp/Topling-288697-9hOD04 2024/11/24-06:27:26.896161 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294920.sst): first pass time = 1.02's, 132.072'MB/sec 2024/11/24-06:27:28.238295 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294920.sst): old prealloc_size = 147639500, real_size = 32914976 2024/11/24-06:27:28.238830 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294920.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929597 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.38 avg-zval = 33.24 2024/11/24-06:27:28.240573 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294920.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:27:29.097273 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294920.sst): second pass time = 1.34's, 80.707'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.100'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3490.642'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 290.904'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33366.644'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.707'MB/sec zip pipeline throughput = 80.707'MB/sec entries = 929597 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.38 avg-zval = 33.24 usrkeys = 929597 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.38 avg-zval = 33.24 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929597 bytes = 3253600 } seq expand size = 6507179 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.4447 value = 3.5010 dict = 1.00 all = 3.6254 } Zip/UnZip{ index = 0.0957 value = 0.2856 dict = 1.00 all = 0.2758 } ---------------------------- total value len = 2252.931397 GB avg = 0.199 KB (by entry num) total key len = 307.247262 GB avg = 0.027 KB total ukey len = 237.614942 GB avg = 0.015 KB total ukey num = 15.751671876 Billion total entry num = 11.292985824 Billion write speed all = 33.018849527 MB/sec (with seq num) write speed all = 31.853676469 MB/sec (without seq num) 2024/11/24-06:27:29.143991 139957399803456 [compaction_job.cc:1948] [default] [JOB 42806] Generated table #294920: 929597 keys, 37021112 bytes, temperature: kWarm 2024/11/24-06:27:29.144065 139957399803456 EVENT_LOG_v1 {"time_micros": 1732400849144027, "cf_name": "default", "job": 42806, "event": "table_file_creation", "file_number": 294920, "file_size": 37021112, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679902, "largest_seqno": 4478439854, "table_properties": {"data_size": 30902352, "index_size": 1780032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26028716, "raw_average_key_size": 28, "raw_value_size": 108189098, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929597, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400845, "oldest_key_time": 0, "file_creation_time": 1732400845, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294920, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:27:29.183577 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294921.sst) /dev/shm/ToplingTemp/Topling-288697-hIhbO5 2024/11/24-06:27:30.142341 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294921.sst): first pass time = 0.96's, 139.996'MB/sec 2024/11/24-06:27:31.408497 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294921.sst): old prealloc_size = 147639500, real_size = 31230592 2024/11/24-06:27:31.409031 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294921.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0292 GB avg-val = 118.00 avg-zval = 31.81 2024/11/24-06:27:31.410590 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294921.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:27:32.219281 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294921.sst): second pass time = 1.27's, 85.725'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.407'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3450.051'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.336'MB/sec write SST data time = 0.13's, 279.721'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 31330.600'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 85.725'MB/sec zip pipeline throughput = 85.725'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.81 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.09 MB all = 0.0352 GB } UnZip/Zip{ index = 10.4455 value = 3.7099 dict = 1.00 all = 3.8138 } Zip/UnZip{ index = 0.0957 value = 0.2696 dict = 1.00 all = 0.2622 } ---------------------------- total value len = 2253.046310 GB avg = 0.199 KB (by entry num) total key len = 307.273003 GB avg = 0.027 KB total ukey len = 237.633328 GB avg = 0.015 KB total ukey num = 15.752591176 Billion total entry num = 11.293905124 Billion write speed all = 33.019334021 MB/sec (with seq num) write speed all = 31.854113031 MB/sec (without seq num) 2024/11/24-06:27:32.262698 139957399803456 [compaction_job.cc:1948] [default] [JOB 42806] Generated table #294921: 919300 keys, 35192434 bytes, temperature: kWarm 2024/11/24-06:27:32.262767 139957399803456 EVENT_LOG_v1 {"time_micros": 1732400852262732, "cf_name": "default", "job": 42806, "event": "table_file_creation", "file_number": 294921, "file_size": 35192434, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4329884199, "largest_seqno": 4441176847, "table_properties": {"data_size": 29240368, "index_size": 1760192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400849, "oldest_key_time": 0, "file_creation_time": 1732400849, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294921, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:27:32.298447 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294922.sst) /dev/shm/ToplingTemp/Topling-288697-7wXlY5 2024/11/24-06:27:33.190323 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294922.sst): first pass time = 0.89's, 150.496'MB/sec 2024/11/24-06:27:34.495535 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294922.sst): old prealloc_size = 147639500, real_size = 31582864 2024/11/24-06:27:34.496134 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294922.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932355 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0296 GB avg-val = 115.96 avg-zval = 31.71 2024/11/24-06:27:34.497548 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294922.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:27:35.329554 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294922.sst): second pass time = 1.30's, 82.878'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.803'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3091.362'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.329'MB/sec write SST data time = 0.13's, 284.746'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34295.175'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 82.878'MB/sec zip pipeline throughput = 82.878'MB/sec entries = 932355 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.96 avg-zval = 31.71 usrkeys = 932355 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.96 avg-zval = 31.71 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932355 bytes = 3263264 } seq expand size = 6526485 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.4475 value = 3.6568 dict = 1.00 all = 3.7599 } Zip/UnZip{ index = 0.0957 value = 0.2735 dict = 1.00 all = 0.2660 } ---------------------------- total value len = 2253.160948 GB avg = 0.199 KB (by entry num) total key len = 307.299109 GB avg = 0.027 KB total ukey len = 237.651975 GB avg = 0.015 KB total ukey num = 15.753523531 Billion total entry num = 11.294837479 Billion write speed all = 33.019824654 MB/sec (with seq num) write speed all = 31.854554211 MB/sec (without seq num) 2024/11/24-06:27:35.374404 139957399803456 [compaction_job.cc:1948] [default] [JOB 42806] Generated table #294922: 932355 keys, 35697038 bytes, temperature: kWarm 2024/11/24-06:27:35.374477 139957399803456 EVENT_LOG_v1 {"time_micros": 1732400855374440, "cf_name": "default", "job": 42806, "event": "table_file_creation", "file_number": 294922, "file_size": 35697038, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679921, "largest_seqno": 4478441160, "table_properties": {"data_size": 29564736, "index_size": 1784832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26105940, "raw_average_key_size": 28, "raw_value_size": 108111858, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932355, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400852, "oldest_key_time": 0, "file_creation_time": 1732400852, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294922, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:27:35.408715 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294923.sst) /dev/shm/ToplingTemp/Topling-288697-vgRmN7 2024/11/24-06:27:36.395334 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294923.sst): first pass time = 0.99's, 136.043'MB/sec 2024/11/24-06:27:37.684250 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294923.sst): old prealloc_size = 147639500, real_size = 31411552 2024/11/24-06:27:37.684788 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294923.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 32.00 2024/11/24-06:27:37.685563 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294923.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:27:38.496865 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294923.sst): second pass time = 1.29's, 84.211'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.531'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3425.209'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.12's, 284.061'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 30089.853'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.211'MB/sec zip pipeline throughput = 84.211'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.00 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4451 value = 3.6870 dict = 1.00 all = 3.7943 } Zip/UnZip{ index = 0.0957 value = 0.2712 dict = 1.00 all = 0.2636 } ---------------------------- total value len = 2253.275861 GB avg = 0.199 KB (by entry num) total key len = 307.324849 GB avg = 0.027 KB total ukey len = 237.670361 GB avg = 0.015 KB total ukey num = 15.754442831 Billion total entry num = 11.295756779 Billion write speed all = 33.020289778 MB/sec (with seq num) write speed all = 31.854972091 MB/sec (without seq num) 2024/11/24-06:27:38.540511 139957399803456 [compaction_job.cc:1948] [default] [JOB 42806] Generated table #294923: 919300 keys, 35373362 bytes, temperature: kWarm 2024/11/24-06:27:38.540580 139957399803456 EVENT_LOG_v1 {"time_micros": 1732400858540545, "cf_name": "default", "job": 42806, "event": "table_file_creation", "file_number": 294923, "file_size": 35373362, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4282843984, "largest_seqno": 4392077168, "table_properties": {"data_size": 29421264, "index_size": 1760256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400855, "oldest_key_time": 0, "file_creation_time": 1732400855, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294923, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:27:38.575527 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294924.sst) /dev/shm/ToplingTemp/Topling-288697-cC1Iv8 2024/11/24-06:27:39.394842 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294924.sst): first pass time = 0.82's, 163.824'MB/sec 2024/11/24-06:27:40.692325 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294924.sst): old prealloc_size = 147639500, real_size = 30898960 2024/11/24-06:27:40.692920 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294924.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925405 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0289 GB avg-val = 117.04 avg-zval = 31.22 2024/11/24-06:27:40.693695 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294924.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:27:41.506426 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294924.sst): second pass time = 1.30's, 83.572'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 42.841'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3108.483'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 277.247'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32996.672'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.572'MB/sec zip pipeline throughput = 83.572'MB/sec entries = 925405 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.04 avg-zval = 31.22 usrkeys = 925405 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.04 avg-zval = 31.22 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925405 bytes = 3123264 } seq expand size = 6477835 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0349 GB } UnZip/Zip{ index = 10.4434 value = 3.7482 dict = 1.00 all = 3.8485 } Zip/UnZip{ index = 0.0958 value = 0.2668 dict = 1.00 all = 0.2598 } ---------------------------- total value len = 2253.390645 GB avg = 0.199 KB (by entry num) total key len = 307.350761 GB avg = 0.027 KB total ukey len = 237.688869 GB avg = 0.015 KB total ukey num = 15.755368236 Billion total entry num = 11.296682184 Billion write speed all = 33.020822593 MB/sec (with seq num) write speed all = 31.855454665 MB/sec (without seq num) 2024/11/24-06:27:41.551234 139957399803456 [compaction_job.cc:1948] [default] [JOB 42806] Generated table #294924: 925405 keys, 34875702 bytes, temperature: kWarm 2024/11/24-06:27:41.551305 139957399803456 EVENT_LOG_v1 {"time_micros": 1732400861551269, "cf_name": "default", "job": 42806, "event": "table_file_creation", "file_number": 294924, "file_size": 34875702, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4392077169, "largest_seqno": 4478441174, "table_properties": {"data_size": 28895232, "index_size": 1772224, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25911340, "raw_average_key_size": 28, "raw_value_size": 108306390, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925405, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400858, "oldest_key_time": 0, "file_creation_time": 1732400858, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294924, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:27:41.588747 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294925.sst) /dev/shm/ToplingTemp/Topling-288697-eob5I6 2024/11/24-06:27:42.546306 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294925.sst): first pass time = 0.96's, 140.173'MB/sec 2024/11/24-06:27:43.879542 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294925.sst): old prealloc_size = 147639500, real_size = 33010784 2024/11/24-06:27:43.880084 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294925.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926235 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 116.91 avg-zval = 33.49 2024/11/24-06:27:43.880845 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294925.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:27:44.728018 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294925.sst): second pass time = 1.33's, 81.268'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.736'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3383.115'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 292.900'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35500.608'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.268'MB/sec zip pipeline throughput = 81.268'MB/sec entries = 926235 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.91 avg-zval = 33.49 usrkeys = 926235 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.91 avg-zval = 33.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926235 bytes = 3241840 } seq expand size = 6483645 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5135 value = 3.4911 dict = 1.00 all = 3.6171 } Zip/UnZip{ index = 0.0951 value = 0.2864 dict = 1.00 all = 0.2765 } ---------------------------- total value len = 2253.505412 GB avg = 0.199 KB (by entry num) total key len = 307.376695 GB avg = 0.027 KB total ukey len = 237.707393 GB avg = 0.015 KB total ukey num = 15.756294471 Billion total entry num = 11.297608419 Billion write speed all = 33.021265161 MB/sec (with seq num) write speed all = 31.855850096 MB/sec (without seq num) 2024/11/24-06:27:44.773756 139957399803456 [compaction_job.cc:1948] [default] [JOB 42806] Generated table #294925: 926235 keys, 37106938 bytes, temperature: kWarm 2024/11/24-06:27:44.773828 139957399803456 EVENT_LOG_v1 {"time_micros": 1732400864773792, "cf_name": "default", "job": 42806, "event": "table_file_creation", "file_number": 294925, "file_size": 37106938, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679941, "largest_seqno": 4478439915, "table_properties": {"data_size": 31017040, "index_size": 1761984, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25934580, "raw_average_key_size": 28, "raw_value_size": 108283258, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926235, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400861, "oldest_key_time": 0, "file_creation_time": 1732400861, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294925, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:27:44.812854 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294926.sst) /dev/shm/ToplingTemp/Topling-288697-To2Lj7 2024/11/24-06:27:45.664949 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294926.sst): first pass time = 0.85's, 157.522'MB/sec 2024/11/24-06:27:46.957147 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294926.sst): old prealloc_size = 147639500, real_size = 31312256 2024/11/24-06:27:46.957674 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294926.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0293 GB avg-val = 118.00 avg-zval = 31.90 2024/11/24-06:27:46.958419 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294926.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:27:47.770342 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294926.sst): second pass time = 1.29's, 84.020'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.379'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3479.538'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 277.430'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 33677.137'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.020'MB/sec zip pipeline throughput = 84.020'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.90 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4458 value = 3.6995 dict = 1.00 all = 3.8050 } Zip/UnZip{ index = 0.0957 value = 0.2703 dict = 1.00 all = 0.2628 } ---------------------------- total value len = 2253.620324 GB avg = 0.199 KB (by entry num) total key len = 307.402436 GB avg = 0.027 KB total ukey len = 237.725779 GB avg = 0.015 KB total ukey num = 15.757213771 Billion total entry num = 11.298527719 Billion write speed all = 33.021783382 MB/sec (with seq num) write speed all = 31.856319206 MB/sec (without seq num) 2024/11/24-06:27:47.814411 139957399803456 [compaction_job.cc:1948] [default] [JOB 42806] Generated table #294926: 919300 keys, 35274098 bytes, temperature: kWarm 2024/11/24-06:27:47.814481 139957399803456 EVENT_LOG_v1 {"time_micros": 1732400867814445, "cf_name": "default", "job": 42806, "event": "table_file_creation", "file_number": 294926, "file_size": 35274098, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4343791367, "largest_seqno": 4455736473, "table_properties": {"data_size": 29322096, "index_size": 1760128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400864, "oldest_key_time": 0, "file_creation_time": 1732400864, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294926, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:27:47.850630 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294927.sst) /dev/shm/ToplingTemp/Topling-288697-MVIhg8 2024/11/24-06:27:48.719781 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294927.sst): first pass time = 0.87's, 154.431'MB/sec 2024/11/24-06:27:50.027068 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294927.sst): old prealloc_size = 147639500, real_size = 32331504 2024/11/24-06:27:50.027610 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294927.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932416 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0303 GB avg-val = 115.95 avg-zval = 32.52 2024/11/24-06:27:50.028408 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294927.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:27:50.876831 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294927.sst): second pass time = 1.31's, 82.750'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 42.891'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3398.772'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 285.932'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31349.696'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.750'MB/sec zip pipeline throughput = 82.750'MB/sec entries = 932416 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.95 avg-zval = 32.52 usrkeys = 932416 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.95 avg-zval = 32.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932416 bytes = 3263472 } seq expand size = 6526912 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5142 value = 3.5651 dict = 1.00 all = 3.6827 } Zip/UnZip{ index = 0.0951 value = 0.2805 dict = 1.00 all = 0.2715 } ---------------------------- total value len = 2253.734961 GB avg = 0.199 KB (by entry num) total key len = 307.428543 GB avg = 0.027 KB total ukey len = 237.744428 GB avg = 0.015 KB total ukey num = 15.758146187 Billion total entry num = 11.299460135 Billion write speed all = 33.022275432 MB/sec (with seq num) write speed all = 31.856761761 MB/sec (without seq num) 2024/11/24-06:27:50.920370 139957399803456 [compaction_job.cc:1948] [default] [JOB 42806] Generated table #294927: 932416 keys, 36445718 bytes, temperature: kWarm 2024/11/24-06:27:50.920451 139957399803456 EVENT_LOG_v1 {"time_micros": 1732400870920413, "cf_name": "default", "job": 42806, "event": "table_file_creation", "file_number": 294927, "file_size": 36445718, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679961, "largest_seqno": 4478441185, "table_properties": {"data_size": 30324576, "index_size": 1773632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26107648, "raw_average_key_size": 28, "raw_value_size": 108110080, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932416, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400867, "oldest_key_time": 0, "file_creation_time": 1732400867, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294927, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:27:50.957953 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294928.sst) /dev/shm/ToplingTemp/Topling-288697-1C7Jw6 2024/11/24-06:27:51.775968 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294928.sst): first pass time = 0.82's, 164.090'MB/sec 2024/11/24-06:27:53.078148 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294928.sst): old prealloc_size = 147639500, real_size = 31484384 2024/11/24-06:27:53.078686 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294928.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.08 2024/11/24-06:27:53.079446 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294928.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:27:53.899896 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294928.sst): second pass time = 1.30's, 83.384'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.136'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3419.685'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.13's, 276.417'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 30431.649'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.384'MB/sec zip pipeline throughput = 83.384'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.92 avg-val = 118.00 avg-zval = 32.08 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.92 avg-val = 125.00 avg-zval = 32.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4436 value = 3.6780 dict = 1.00 all = 3.7865 } Zip/UnZip{ index = 0.0958 value = 0.2719 dict = 1.00 all = 0.2641 } ---------------------------- total value len = 2253.849874 GB avg = 0.199 KB (by entry num) total key len = 307.454284 GB avg = 0.027 KB total ukey len = 237.762814 GB avg = 0.015 KB total ukey num = 15.759065487 Billion total entry num = 11.300379435 Billion write speed all = 33.022801779 MB/sec (with seq num) write speed all = 31.857238716 MB/sec (without seq num) 2024/11/24-06:27:53.942745 139957399803456 [compaction_job.cc:1948] [default] [JOB 42806] Generated table #294928: 919300 keys, 35446194 bytes, temperature: kWarm 2024/11/24-06:27:53.942815 139957399803456 EVENT_LOG_v1 {"time_micros": 1732400873942778, "cf_name": "default", "job": 42806, "event": "table_file_creation", "file_number": 294928, "file_size": 35446194, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4296302945, "largest_seqno": 4406469302, "table_properties": {"data_size": 29493840, "index_size": 1760512, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400870, "oldest_key_time": 0, "file_creation_time": 1732400870, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294928, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:27:53.979248 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294929.sst) /dev/shm/ToplingTemp/Topling-288697-Gm61s5 2024/11/24-06:27:54.928774 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294929.sst): first pass time = 0.95's, 141.359'MB/sec 2024/11/24-06:27:56.221332 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294929.sst): old prealloc_size = 147639500, real_size = 30426784 2024/11/24-06:27:56.221908 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294929.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928681 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0284 GB avg-val = 116.53 avg-zval = 30.59 2024/11/24-06:27:56.223874 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294929.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:27:57.037208 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294929.sst): second pass time = 1.29's, 83.774'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.181'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3233.409'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.337'MB/sec write SST data time = 0.13's, 270.741'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 27885.036'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.774'MB/sec zip pipeline throughput = 83.774'MB/sec entries = 928681 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.53 avg-zval = 30.59 usrkeys = 928681 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.53 avg-zval = 30.59 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928681 bytes = 3134320 } seq expand size = 6500767 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0284 GB dict = 1.08 MB all = 0.0344 GB } UnZip/Zip{ index = 10.3777 value = 3.8097 dict = 1.00 all = 3.9002 } Zip/UnZip{ index = 0.0964 value = 0.2625 dict = 1.00 all = 0.2564 } ---------------------------- total value len = 2253.964589 GB avg = 0.199 KB (by entry num) total key len = 307.480287 GB avg = 0.027 KB total ukey len = 237.781387 GB avg = 0.015 KB total ukey num = 15.759994168 Billion total entry num = 11.301308116 Billion write speed all = 33.023280293 MB/sec (with seq num) write speed all = 31.857668590 MB/sec (without seq num) 2024/11/24-06:27:57.079129 139957399803456 [compaction_job.cc:1948] [default] [JOB 42806] Generated table #294929: 928681 keys, 34412728 bytes, temperature: kWarm 2024/11/24-06:27:57.079196 139957399803456 EVENT_LOG_v1 {"time_micros": 1732400877079160, "cf_name": "default", "job": 42806, "event": "table_file_creation", "file_number": 294929, "file_size": 34412728, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4406469303, "largest_seqno": 4478441205, "table_properties": {"data_size": 28404688, "index_size": 1789760, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26003068, "raw_average_key_size": 28, "raw_value_size": 108214742, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928681, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400873, "oldest_key_time": 0, "file_creation_time": 1732400873, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294929, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:27:57.113657 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294930.sst) /dev/shm/ToplingTemp/Topling-288697-6irsq8 2024/11/24-06:27:58.119015 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294930.sst): first pass time = 1.01's, 133.508'MB/sec 2024/11/24-06:27:59.480522 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294930.sst): old prealloc_size = 147639500, real_size = 32570560 2024/11/24-06:27:59.481051 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294930.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923104 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0306 GB avg-val = 117.40 avg-zval = 33.13 2024/11/24-06:27:59.483034 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294930.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:28:00.315713 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294930.sst): second pass time = 1.36's, 79.675'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.895'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3481.062'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 290.682'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 33902.910'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.675'MB/sec zip pipeline throughput = 79.675'MB/sec entries = 923104 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.40 avg-zval = 33.13 usrkeys = 923104 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.40 avg-zval = 33.13 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923104 bytes = 3230880 } seq expand size = 6461728 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5154 value = 3.5434 dict = 1.00 all = 3.6612 } Zip/UnZip{ index = 0.0951 value = 0.2822 dict = 1.00 all = 0.2731 } ---------------------------- total value len = 2254.079422 GB avg = 0.199 KB (by entry num) total key len = 307.506134 GB avg = 0.027 KB total ukey len = 237.799850 GB avg = 0.015 KB total ukey num = 15.760917272 Billion total entry num = 11.302231220 Billion write speed all = 33.023698153 MB/sec (with seq num) write speed all = 31.858040511 MB/sec (without seq num) 2024/11/24-06:28:00.360564 139957399803456 [compaction_job.cc:1948] [default] [JOB 42806] Generated table #294930: 923104 keys, 36659014 bytes, temperature: kWarm 2024/11/24-06:28:00.360635 139957399803456 EVENT_LOG_v1 {"time_micros": 1732400880360598, "cf_name": "default", "job": 42806, "event": "table_file_creation", "file_number": 294930, "file_size": 36659014, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679976, "largest_seqno": 4478440381, "table_properties": {"data_size": 30583920, "index_size": 1755712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25846912, "raw_average_key_size": 28, "raw_value_size": 108370872, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923104, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400877, "oldest_key_time": 0, "file_creation_time": 1732400877, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294930, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:28:00.395886 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294931.sst) /dev/shm/ToplingTemp/Topling-288697-nxTzM7 2024/11/24-06:28:01.394575 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294931.sst): first pass time = 1.00's, 134.400'MB/sec 2024/11/24-06:28:02.716798 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294931.sst): old prealloc_size = 147639500, real_size = 31331232 2024/11/24-06:28:02.717348 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294931.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0293 GB avg-val = 118.00 avg-zval = 31.92 2024/11/24-06:28:02.719348 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294931.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:28:03.537600 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294931.sst): second pass time = 1.32's, 82.095'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.999'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3349.253'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.13's, 272.834'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 34009.654'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.095'MB/sec zip pipeline throughput = 82.095'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.92 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4439 value = 3.6972 dict = 1.00 all = 3.8030 } Zip/UnZip{ index = 0.0957 value = 0.2705 dict = 1.00 all = 0.2630 } ---------------------------- total value len = 2254.194334 GB avg = 0.199 KB (by entry num) total key len = 307.531874 GB avg = 0.027 KB total ukey len = 237.818236 GB avg = 0.015 KB total ukey num = 15.761836572 Billion total entry num = 11.303150520 Billion write speed all = 33.024139734 MB/sec (with seq num) write speed all = 31.858435699 MB/sec (without seq num) 2024/11/24-06:28:03.581917 139957399803456 [compaction_job.cc:1948] [default] [JOB 42806] Generated table #294931: 919300 keys, 35293042 bytes, temperature: kWarm 2024/11/24-06:28:03.582002 139957399803456 EVENT_LOG_v1 {"time_micros": 1732400883581952, "cf_name": "default", "job": 42806, "event": "table_file_creation", "file_number": 294931, "file_size": 35293042, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4357122895, "largest_seqno": 4469579147, "table_properties": {"data_size": 29340752, "index_size": 1760448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400880, "oldest_key_time": 0, "file_creation_time": 1732400880, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294931, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:28:03.618232 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294932.sst) /dev/shm/ToplingTemp/Topling-288697-u5Inj7 2024/11/24-06:28:04.058476 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294932.sst): first pass time = 0.44's, 171.892'MB/sec 2024/11/24-06:28:04.803651 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294932.sst): old prealloc_size = 147639500, real_size = 17742592 2024/11/24-06:28:04.804002 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294932.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 530851 prefix = 4 raw-key = 0.0085 GB zip-key = 0.0010 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0608 GB zip-val = 0.0166 GB avg-val = 114.54 avg-zval = 31.26 2024/11/24-06:28:04.804478 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294932.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0013 GB, waited 0.000 sec, Key+Value = 0.076 GB 2024/11/24-06:28:05.296049 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00/294932.sst): second pass time = 0.74's, 81.676'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.49's, 40.517'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3054.392'MB/sec (index lex order gen) rebuild zvType time = 0.42's, 0.318'MB/sec write SST data time = 0.08's, 264.896'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.611'MB, throughput = 19864.239'MB/sec zip my value time = 0.74's, unzip length = 0.061'GB zip my value throughput = 81.676'MB/sec zip pipeline throughput = 81.676'MB/sec entries = 530851 avg-key = 28.00 avg-zkey = 1.92 avg-val = 114.54 avg-zval = 31.26 usrkeys = 530851 avg-key = 20.00 avg-zkey = 1.92 avg-val = 121.54 avg-zval = 31.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 530851 bytes = 1725280 } seq expand size = 3715957 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0106 GB value = 0.0608 GB dict = 0.61 MB all = 0.0757 GB } __ZipSize{ index = 0.0010 GB value = 0.0166 GB dict = 0.61 MB all = 0.0199 GB } UnZip/Zip{ index = 10.4393 value = 3.6646 dict = 1.00 all = 3.7931 } Zip/UnZip{ index = 0.0958 value = 0.2729 dict = 1.00 all = 0.2636 } ---------------------------- total value len = 2254.258855 GB avg = 0.199 KB (by entry num) total key len = 307.546738 GB avg = 0.027 KB total ukey len = 237.828853 GB avg = 0.015 KB total ukey num = 15.762367423 Billion total entry num = 11.303681371 Billion write speed all = 33.024414497 MB/sec (with seq num) write speed all = 31.858682141 MB/sec (without seq num) 2024/11/24-06:28:05.325958 139957399803456 [compaction_job.cc:1948] [default] [JOB 42806] Generated table #294932: 530851 keys, 19948959 bytes, temperature: kWarm 2024/11/24-06:28:05.326046 139957399803456 EVENT_LOG_v1 {"time_micros": 1732400885326009, "cf_name": "default", "job": 42806, "event": "table_file_creation", "file_number": 294932, "file_size": 19948959, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4427138413, "largest_seqno": 4478441215, "table_properties": {"data_size": 16592688, "index_size": 1017024, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 14863828, "raw_average_key_size": 28, "raw_value_size": 60804930, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 530851, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400883, "oldest_key_time": 0, "file_creation_time": 1732400883, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294932, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:28:05.345163 139957399803456 [compaction_job.cc:703] job-42806: subcompact[0], size: 0.902838 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42806/att-00 [294907,294908,294909,294910,294911,294912,294913,294914,294915,294916,294917,294918,294919,294920,294921,294922,294923,294924,294925,294926,294927,294928,294929,294930,294931,294932] 2024/11/24-06:28:05.492484 139957399803456 (Original Log Time 2024/11/24-06:26:38.774310) EVENT_LOG_v1 {"time_micros": 1732400798774270, "job": 42806, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [294899], "files_L3": [294869, 294870, 294871, 294825, 294826, 294827, 294828, 294829, 294830, 294831, 294832, 294833, 294834, 294835, 294836, 294837, 294838, 294839, 294840, 294841, 294842, 294843, 294844, 294845, 294846, 294847, 294848, 294797], "score": 0.986433, "input_data_size": 927830652, "oldest_snapshot_seqno": 4478453667} 2024/11/24-06:28:05.492487 139957399803456 (Original Log Time 2024/11/24-06:28:05.490794) [compaction_job.cc:2022] [default] [JOB 42806] Compacted 1@2 + 28@3 files to L3 => 902838149 bytes 2024/11/24-06:28:05.492488 139957399803456 (Original Log Time 2024/11/24-06:28:05.492401) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 26 0 0 0] max score 0.03, MB/sec: 10.7 rd, 10.4 wr, level 3, files in(1, 28) out(26 +0 blob) MB in(27.1, 857.8 +0.0 blob) out(861.0 +0.0 blob), read-write-amplify(64.5) write-amplify(31.8) OK, records in: 23808663, records dropped: 102403 output_compression: Snappy 2024/11/24-06:28:05.492492 139957399803456 (Original Log Time 2024/11/24-06:28:05.492456) EVENT_LOG_v1 {"time_micros": 1732400885492427, "cf": "default", "job": 42806, "event": "compaction_finished", "compaction_time_micros": 86570860, "compaction_time_cpu_micros": 36156069, "output_level": 3, "num_output_files": 26, "total_output_size": 902838149, "num_input_records": 23808663, "num_output_records": 23706260, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 662797507, "file_range_sync_nanos": 0, "file_fsync_nanos": 20334, "file_prepare_write_nanos": 74840, "lsm_state": [0, 0, 0, 26, 0, 0, 0]} 2024/11/24-06:28:05.636959 139957399803456 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732400798,"executesMs":86860,"compactionJobId":42806,"attempt":0,"compactionInputRawBytes":3445654846,"compactionInputZipBytes":927830652,"compactionOutputRawBytes":0,"compactionOutputZipBytes":902838149,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-e7aaf5db6dd9aa1e40ae8099446981cb-680fe336a118cf19-00"}}, response = 2024/11/24-06:28:05.637103 139957399803456 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-42806/att-00: olev 3, work 86.861 s, result 6.023 ms, install 1.714 ms, input{raw 3.209 GiB zip 884.848 MiB}