2024/11/24-06:03:52.298730 139957609621056 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00: fork to child time = 0.018373 sec 2024/11/24-06:03:52.299393 139957609621056 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00 2024/11/24-06:03:52.305201 139957609621056 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00 2024/11/24-06:03:52.306103 139957609621056 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-06:03:52.306360 139957609621056 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-06:03:52.361619 139957609621056 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-42573/att-00/MANIFEST-206347 2024/11/24-06:03:52.376691 139957609621056 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-42573/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 294389, last_sequence is 4471362767, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 294213 2024/11/24-06:03:52.376697 139957609621056 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-06:03:52.376814 139957609621056 [version_set.cc:5721] Creating manifest 294389 2024/11/24-06:03:52.410863 139957609621056 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-06:03:52.430814 139957609621056 [compaction_job.cc:2437] [default] [JOB 42573] Compacting 1@2 + 23@3 files to L3, score 0.95, subcompactions 1 : 1 2024/11/24-06:03:52.430830 139957609621056 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 31919 Base level 2, inputs: [294347(26MB)], [294387(29MB) 294290(33MB) 294291(31MB) 294292(35MB) 294293(33MB) 294294(32MB) 294295(33MB) 294296(33MB) 294297(35MB) 294298(33MB) 294299(35MB) 294300(33MB) 294301(32MB) 294302(33MB) 294303(33MB) 294304(35MB) 294305(33MB) 294306(34MB) 294307(20MB) 294264(33MB) 294265(35MB) 294266(33MB) 294267(26MB)] 2024/11/24-06:03:52.439451 139957609621056 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-06:03:52.439483 139957609621056 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-06:03:52.439499 139957609621056 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-06:03:52.439504 139957609621056 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-06:03:52.439516 139957609621056 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-06:03:52.932063 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294390.sst) /dev/shm/ToplingTemp/Topling-285412-PGZ2jG 2024/11/24-06:03:54.021031 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294390.sst): first pass time = 1.09's, 123.257'MB/sec 2024/11/24-06:03:56.125215 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294390.sst): old prealloc_size = 147639500, real_size = 30983008 2024/11/24-06:03:56.125815 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294390.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 938921 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1079 GB zip-val = 0.0290 GB avg-val = 114.95 avg-zval = 30.85 2024/11/24-06:03:56.126640 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294390.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:03:57.242177 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294390.sst): second pass time = 2.10's, 51.314'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.12's, 31.440'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3098.473'MB/sec (index lex order gen) rebuild zvType time = 0.95's, 0.247'MB/sec write SST data time = 0.17's, 212.022'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 34623.260'MB/sec zip my value time = 2.10's, unzip length = 0.108'GB zip my value throughput = 51.314'MB/sec zip pipeline throughput = 56.762'MB/sec entries = 938921 avg-key = 28.00 avg-zkey = 1.90 avg-val = 114.95 avg-zval = 30.85 usrkeys = 938921 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.95 avg-zval = 30.85 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 938921 bytes = 3286240 } seq expand size = 6572447 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0188 GB value = 0.1079 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0290 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.5124 value = 3.7266 dict = 1.00 all = 3.8220 } Zip/UnZip{ index = 0.0951 value = 0.2683 dict = 1.00 all = 0.2616 } ---------------------------- total value len = 2251.440074 GB avg = 0.200 KB (by entry num) total key len = 306.909252 GB avg = 0.027 KB total ukey len = 237.373506 GB avg = 0.015 KB total ukey num = 15.739600084 Billion total entry num = 11.280914032 Billion write speed all = 33.607203952 MB/sec (with seq num) write speed all = 32.421689605 MB/sec (without seq num) 2024/11/24-06:03:57.289251 139957609621056 [compaction_job.cc:1948] [default] [JOB 42573] Generated table #294390: 938921 keys, 35117218 bytes, temperature: kWarm 2024/11/24-06:03:57.289379 139957609621056 EVENT_LOG_v1 {"time_micros": 1732399437289318, "cf_name": "default", "job": 42573, "event": "table_file_creation", "file_number": 294390, "file_size": 35117218, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4470945020, "table_properties": {"data_size": 28961808, "index_size": 1786304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26289788, "raw_average_key_size": 28, "raw_value_size": 107927986, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 938921, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399432, "oldest_key_time": 0, "file_creation_time": 1732399432, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294390, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:03:57.325650 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294391.sst) /dev/shm/ToplingTemp/Topling-285412-qAzDnK 2024/11/24-06:03:58.273364 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294391.sst): first pass time = 0.95's, 141.629'MB/sec 2024/11/24-06:04:00.048139 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294391.sst): old prealloc_size = 147639500, real_size = 31415152 2024/11/24-06:04:00.048693 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294391.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 32.01 2024/11/24-06:04:00.049481 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294391.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:04:01.092689 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294391.sst): second pass time = 1.77's, 61.167'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.04's, 33.979'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3292.071'MB/sec (index lex order gen) rebuild zvType time = 0.89's, 0.259'MB/sec write SST data time = 0.16's, 225.087'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 34457.889'MB/sec zip my value time = 1.77's, unzip length = 0.108'GB zip my value throughput = 61.167'MB/sec zip pipeline throughput = 61.167'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.01 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.01 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4439 value = 3.6866 dict = 1.00 all = 3.7816 } Zip/UnZip{ index = 0.0957 value = 0.2713 dict = 1.00 all = 0.2644 } ---------------------------- total value len = 2251.554986 GB avg = 0.200 KB (by entry num) total key len = 306.934993 GB avg = 0.027 KB total ukey len = 237.391892 GB avg = 0.015 KB total ukey num = 15.740519384 Billion total entry num = 11.281833332 Billion write speed all = 33.607351702 MB/sec (with seq num) write speed all = 32.421800713 MB/sec (without seq num) 2024/11/24-06:04:01.138045 139957609621056 [compaction_job.cc:1948] [default] [JOB 42573] Generated table #294391: 919300 keys, 35491890 bytes, temperature: kWarm 2024/11/24-06:04:01.138115 139957609621056 EVENT_LOG_v1 {"time_micros": 1732399441138080, "cf_name": "default", "job": 42573, "event": "table_file_creation", "file_number": 294391, "file_size": 35491890, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4298115069, "largest_seqno": 4445356498, "table_properties": {"data_size": 29424672, "index_size": 1760448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399437, "oldest_key_time": 0, "file_creation_time": 1732399437, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294391, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:04:01.175176 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294392.sst) /dev/shm/ToplingTemp/Topling-285412-wfSC9I 2024/11/24-06:04:02.204825 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294392.sst): first pass time = 1.03's, 130.359'MB/sec 2024/11/24-06:04:03.787098 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294392.sst): old prealloc_size = 147639500, real_size = 28510128 2024/11/24-06:04:03.787702 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294392.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 941001 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1079 GB zip-val = 0.0265 GB avg-val = 114.63 avg-zval = 28.12 2024/11/24-06:04:03.788478 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294392.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:04:04.617053 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294392.sst): second pass time = 1.58's, 68.211'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 39.198'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3101.869'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.336'MB/sec write SST data time = 0.13's, 251.242'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.076'MB, throughput = 27625.953'MB/sec zip my value time = 1.58's, unzip length = 0.108'GB zip my value throughput = 68.211'MB/sec zip pipeline throughput = 126.913'MB/sec entries = 941001 avg-key = 28.00 avg-zkey = 1.93 avg-val = 114.63 avg-zval = 28.12 usrkeys = 941001 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.63 avg-zval = 28.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 941001 bytes = 3175888 } seq expand size = 6587007 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0188 GB value = 0.1079 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0265 GB dict = 1.08 MB all = 0.0325 GB } UnZip/Zip{ index = 10.3759 value = 4.0766 dict = 1.00 all = 4.1258 } Zip/UnZip{ index = 0.0964 value = 0.2453 dict = 1.00 all = 0.2424 } ---------------------------- total value len = 2251.669443 GB avg = 0.200 KB (by entry num) total key len = 306.961341 GB avg = 0.027 KB total ukey len = 237.410712 GB avg = 0.015 KB total ukey num = 15.741460385 Billion total entry num = 11.282774333 Billion write speed all = 33.607645405 MB/sec (with seq num) write speed all = 32.422050418 MB/sec (without seq num) 2024/11/24-06:04:04.665689 139957609621056 [compaction_job.cc:1948] [default] [JOB 42573] Generated table #294392: 941001 keys, 32531254 bytes, temperature: kWarm 2024/11/24-06:04:04.665756 139957609621056 EVENT_LOG_v1 {"time_micros": 1732399444665721, "cf_name": "default", "job": 42573, "event": "table_file_creation", "file_number": 294392, "file_size": 32531254, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4344191945, "largest_seqno": 4470947734, "table_properties": {"data_size": 26460896, "index_size": 1813824, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26348028, "raw_average_key_size": 28, "raw_value_size": 107869774, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 941001, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399441, "oldest_key_time": 0, "file_creation_time": 1732399441, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294392, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:04:04.702571 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294393.sst) /dev/shm/ToplingTemp/Topling-285412-9VCIPG 2024/11/24-06:04:05.543439 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294393.sst): first pass time = 0.84's, 159.625'MB/sec 2024/11/24-06:04:07.104221 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294393.sst): old prealloc_size = 147639500, real_size = 32884352 2024/11/24-06:04:07.104799 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294393.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929464 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.40 avg-zval = 33.23 2024/11/24-06:04:07.105587 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294393.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:04:07.968409 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294393.sst): second pass time = 1.56's, 69.393'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.805'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3205.925'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.316'MB/sec write SST data time = 0.13's, 287.524'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32985.725'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.393'MB/sec zip pipeline throughput = 98.899'MB/sec entries = 929464 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.40 avg-zval = 33.23 usrkeys = 929464 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.40 avg-zval = 33.23 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929464 bytes = 3253136 } seq expand size = 6506248 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5151 value = 3.5032 dict = 1.00 all = 3.6285 } Zip/UnZip{ index = 0.0951 value = 0.2855 dict = 1.00 all = 0.2756 } ---------------------------- total value len = 2251.784142 GB avg = 0.200 KB (by entry num) total key len = 306.987366 GB avg = 0.027 KB total ukey len = 237.429301 GB avg = 0.015 KB total ukey num = 15.742389849 Billion total entry num = 11.283703797 Billion write speed all = 33.608014391 MB/sec (with seq num) write speed all = 32.422373928 MB/sec (without seq num) 2024/11/24-06:04:08.014306 139957609621056 [compaction_job.cc:1948] [default] [JOB 42573] Generated table #294393: 929464 keys, 36990120 bytes, temperature: kWarm 2024/11/24-06:04:08.014387 139957609621056 EVENT_LOG_v1 {"time_micros": 1732399448014340, "cf_name": "default", "job": 42573, "event": "table_file_creation", "file_number": 294393, "file_size": 36990120, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679865, "largest_seqno": 4470948009, "table_properties": {"data_size": 30883952, "index_size": 1767872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26024992, "raw_average_key_size": 28, "raw_value_size": 108192820, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929464, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399444, "oldest_key_time": 0, "file_creation_time": 1732399444, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294393, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:04:08.051863 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294394.sst) /dev/shm/ToplingTemp/Topling-285412-R8h1cJ 2024/11/24-06:04:08.972872 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294394.sst): first pass time = 0.92's, 145.735'MB/sec 2024/11/24-06:04:10.295347 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294394.sst): old prealloc_size = 147639500, real_size = 31481648 2024/11/24-06:04:10.295915 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294394.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.08 2024/11/24-06:04:10.296690 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294394.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:04:11.115253 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294394.sst): second pass time = 1.32's, 82.111'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.230'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3300.990'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.13's, 279.045'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 27401.318'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.111'MB/sec zip pipeline throughput = 82.111'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.08 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4451 value = 3.6783 dict = 1.00 all = 3.7868 } Zip/UnZip{ index = 0.0957 value = 0.2719 dict = 1.00 all = 0.2641 } ---------------------------- total value len = 2251.899054 GB avg = 0.200 KB (by entry num) total key len = 307.013106 GB avg = 0.027 KB total ukey len = 237.447687 GB avg = 0.015 KB total ukey num = 15.743309149 Billion total entry num = 11.284623097 Billion write speed all = 33.608472680 MB/sec (with seq num) write speed all = 32.422784627 MB/sec (without seq num) 2024/11/24-06:04:11.158994 139957609621056 [compaction_job.cc:1948] [default] [JOB 42573] Generated table #294394: 919300 keys, 35443506 bytes, temperature: kWarm 2024/11/24-06:04:11.159063 139957609621056 EVENT_LOG_v1 {"time_micros": 1732399451159027, "cf_name": "default", "job": 42573, "event": "table_file_creation", "file_number": 294394, "file_size": 35443506, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4334993442, "largest_seqno": 4454734997, "table_properties": {"data_size": 29491360, "index_size": 1760256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399448, "oldest_key_time": 0, "file_creation_time": 1732399448, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294394, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:04:11.200738 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294395.sst) /dev/shm/ToplingTemp/Topling-285412-ErWYaG 2024/11/24-06:04:12.177017 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294395.sst): first pass time = 0.98's, 137.483'MB/sec 2024/11/24-06:04:13.518020 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294395.sst): old prealloc_size = 147639500, real_size = 30953712 2024/11/24-06:04:13.518627 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294395.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 937196 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0289 GB avg-val = 115.21 avg-zval = 30.87 2024/11/24-06:04:13.519419 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294395.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:04:14.362290 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294395.sst): second pass time = 1.34's, 80.572'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 41.554'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3066.722'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.327'MB/sec write SST data time = 0.13's, 274.522'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 31363.830'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.572'MB/sec zip pipeline throughput = 80.572'MB/sec entries = 937196 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.21 avg-zval = 30.87 usrkeys = 937196 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.21 avg-zval = 30.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 937196 bytes = 3280208 } seq expand size = 6560372 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.5108 value = 3.7316 dict = 1.00 all = 3.8258 } Zip/UnZip{ index = 0.0951 value = 0.2680 dict = 1.00 all = 0.2614 } ---------------------------- total value len = 2252.013591 GB avg = 0.200 KB (by entry num) total key len = 307.039348 GB avg = 0.027 KB total ukey len = 237.466431 GB avg = 0.015 KB total ukey num = 15.744246345 Billion total entry num = 11.285560293 Billion write speed all = 33.608888353 MB/sec (with seq num) write speed all = 32.423152395 MB/sec (without seq num) 2024/11/24-06:04:14.406484 139957609621056 [compaction_job.cc:1948] [default] [JOB 42573] Generated table #294395: 937196 keys, 35082648 bytes, temperature: kWarm 2024/11/24-06:04:14.406554 139957609621056 EVENT_LOG_v1 {"time_micros": 1732399454406518, "cf_name": "default", "job": 42573, "event": "table_file_creation", "file_number": 294395, "file_size": 35082648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679880, "largest_seqno": 4470947744, "table_properties": {"data_size": 28935904, "index_size": 1783296, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26241488, "raw_average_key_size": 28, "raw_value_size": 107976288, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 937196, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399451, "oldest_key_time": 0, "file_creation_time": 1732399451, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294395, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:04:14.444946 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294396.sst) /dev/shm/ToplingTemp/Topling-285412-V5qVBG 2024/11/24-06:04:15.440233 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294396.sst): first pass time = 1.00's, 134.861'MB/sec 2024/11/24-06:04:16.738878 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294396.sst): old prealloc_size = 147639500, real_size = 31434720 2024/11/24-06:04:16.739445 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294396.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 32.03 2024/11/24-06:04:16.740187 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294396.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:04:17.553802 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294396.sst): second pass time = 1.30's, 83.616'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.437'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3251.452'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 278.125'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 31742.145'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.616'MB/sec zip pipeline throughput = 83.616'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.03 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4447 value = 3.6841 dict = 1.00 all = 3.7918 } Zip/UnZip{ index = 0.0957 value = 0.2714 dict = 1.00 all = 0.2637 } ---------------------------- total value len = 2252.128504 GB avg = 0.200 KB (by entry num) total key len = 307.065088 GB avg = 0.027 KB total ukey len = 237.484817 GB avg = 0.015 KB total ukey num = 15.745165645 Billion total entry num = 11.286479593 Billion write speed all = 33.609326852 MB/sec (with seq num) write speed all = 32.423544009 MB/sec (without seq num) 2024/11/24-06:04:17.598681 139957609621056 [compaction_job.cc:1948] [default] [JOB 42573] Generated table #294396: 919300 keys, 35396530 bytes, temperature: kWarm 2024/11/24-06:04:17.598753 139957609621056 EVENT_LOG_v1 {"time_micros": 1732399457598718, "cf_name": "default", "job": 42573, "event": "table_file_creation", "file_number": 294396, "file_size": 35396530, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4284367024, "largest_seqno": 4394821700, "table_properties": {"data_size": 29444368, "index_size": 1760320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399454, "oldest_key_time": 0, "file_creation_time": 1732399454, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294396, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:04:17.638390 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294397.sst) /dev/shm/ToplingTemp/Topling-285412-kTljgK 2024/11/24-06:04:18.670236 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294397.sst): first pass time = 1.03's, 130.080'MB/sec 2024/11/24-06:04:19.966092 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294397.sst): old prealloc_size = 147639500, real_size = 30584032 2024/11/24-06:04:19.966692 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294397.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927855 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0286 GB avg-val = 116.65 avg-zval = 30.80 2024/11/24-06:04:19.968682 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294397.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:04:20.784238 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294397.sst): second pass time = 1.30's, 83.580'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.253'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3089.121'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 269.925'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33928.630'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.580'MB/sec zip pipeline throughput = 83.580'MB/sec entries = 927855 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.65 avg-zval = 30.80 usrkeys = 927855 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.65 avg-zval = 30.80 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927855 bytes = 3131520 } seq expand size = 6494985 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0286 GB dict = 1.08 MB all = 0.0346 GB } UnZip/Zip{ index = 10.4451 value = 3.7878 dict = 1.00 all = 3.8828 } Zip/UnZip{ index = 0.0957 value = 0.2640 dict = 1.00 all = 0.2575 } ---------------------------- total value len = 2252.243236 GB avg = 0.200 KB (by entry num) total key len = 307.091068 GB avg = 0.027 KB total ukey len = 237.503374 GB avg = 0.015 KB total ukey num = 15.746093500 Billion total entry num = 11.287407448 Billion write speed all = 33.609748921 MB/sec (with seq num) write speed all = 32.423918904 MB/sec (without seq num) 2024/11/24-06:04:20.828420 139957609621056 [compaction_job.cc:1948] [default] [JOB 42573] Generated table #294397: 927855 keys, 34567652 bytes, temperature: kWarm 2024/11/24-06:04:20.828512 139957609621056 EVENT_LOG_v1 {"time_micros": 1732399460828462, "cf_name": "default", "job": 42573, "event": "table_file_creation", "file_number": 294397, "file_size": 34567652, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4394821701, "largest_seqno": 4470947763, "table_properties": {"data_size": 28575248, "index_size": 1776640, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25979940, "raw_average_key_size": 28, "raw_value_size": 108237822, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927855, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399457, "oldest_key_time": 0, "file_creation_time": 1732399457, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294397, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:04:20.865383 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294398.sst) /dev/shm/ToplingTemp/Topling-285412-Yx8hUJ 2024/11/24-06:04:21.867037 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294398.sst): first pass time = 1.00's, 134.001'MB/sec 2024/11/24-06:04:23.175257 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294398.sst): old prealloc_size = 147639500, real_size = 32886752 2024/11/24-06:04:23.175791 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294398.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924544 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 117.17 avg-zval = 33.42 2024/11/24-06:04:23.177862 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294398.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:04:24.026064 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294398.sst): second pass time = 1.31's, 82.857'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.464'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3460.099'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 287.765'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33683.023'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.857'MB/sec zip pipeline throughput = 82.857'MB/sec entries = 924544 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.17 avg-zval = 33.42 usrkeys = 924544 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.17 avg-zval = 33.42 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924544 bytes = 3235920 } seq expand size = 6471808 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5123 value = 3.5062 dict = 1.00 all = 3.6296 } Zip/UnZip{ index = 0.0951 value = 0.2852 dict = 1.00 all = 0.2755 } ---------------------------- total value len = 2252.358039 GB avg = 0.200 KB (by entry num) total key len = 307.116955 GB avg = 0.027 KB total ukey len = 237.521865 GB avg = 0.015 KB total ukey num = 15.747018044 Billion total entry num = 11.288331992 Billion write speed all = 33.610165622 MB/sec (with seq num) write speed all = 32.424288960 MB/sec (without seq num) 2024/11/24-06:04:24.071722 139957609621056 [compaction_job.cc:1948] [default] [JOB 42573] Generated table #294398: 924544 keys, 36978484 bytes, temperature: kWarm 2024/11/24-06:04:24.071792 139957609621056 EVENT_LOG_v1 {"time_micros": 1732399464071756, "cf_name": "default", "job": 42573, "event": "table_file_creation", "file_number": 294398, "file_size": 36978484, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679891, "largest_seqno": 4470945906, "table_properties": {"data_size": 30896464, "index_size": 1758976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25887232, "raw_average_key_size": 28, "raw_value_size": 108330552, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924544, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399460, "oldest_key_time": 0, "file_creation_time": 1732399460, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294398, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:04:24.114127 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294399.sst) /dev/shm/ToplingTemp/Topling-285412-BsahwG 2024/11/24-06:04:25.070004 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294399.sst): first pass time = 0.96's, 140.419'MB/sec 2024/11/24-06:04:26.382545 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294399.sst): old prealloc_size = 147639500, real_size = 31350896 2024/11/24-06:04:26.383212 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294399.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1085 GB zip-val = 0.0293 GB avg-val = 118.00 avg-zval = 31.93 2024/11/24-06:04:26.385233 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294399.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:04:27.206640 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294399.sst): second pass time = 1.31's, 82.697'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.852'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2710.217'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.13's, 271.103'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 35194.460'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.697'MB/sec zip pipeline throughput = 82.697'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 31.93 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 31.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.3775 value = 3.6961 dict = 1.00 all = 3.8008 } Zip/UnZip{ index = 0.0964 value = 0.2706 dict = 1.00 all = 0.2631 } ---------------------------- total value len = 2252.472951 GB avg = 0.200 KB (by entry num) total key len = 307.142696 GB avg = 0.027 KB total ukey len = 237.540251 GB avg = 0.015 KB total ukey num = 15.747937344 Billion total entry num = 11.289251292 Billion write speed all = 33.610608840 MB/sec (with seq num) write speed all = 32.424685134 MB/sec (without seq num) 2024/11/24-06:04:27.251851 139957609621056 [compaction_job.cc:1948] [default] [JOB 42573] Generated table #294399: 919300 keys, 35312690 bytes, temperature: kWarm 2024/11/24-06:04:27.251922 139957609621056 EVENT_LOG_v1 {"time_micros": 1732399467251886, "cf_name": "default", "job": 42573, "event": "table_file_creation", "file_number": 294399, "file_size": 35312690, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4351055958, "largest_seqno": 4463137571, "table_properties": {"data_size": 29349152, "index_size": 1771712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399464, "oldest_key_time": 0, "file_creation_time": 1732399464, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294399, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:04:27.291128 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294400.sst) /dev/shm/ToplingTemp/Topling-285412-T1KzwK 2024/11/24-06:04:28.275862 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294400.sst): first pass time = 0.98's, 136.304'MB/sec 2024/11/24-06:04:29.617763 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294400.sst): old prealloc_size = 147639500, real_size = 32821568 2024/11/24-06:04:29.618311 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294400.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932047 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 116.00 avg-zval = 33.06 2024/11/24-06:04:29.620382 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294400.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:04:30.480911 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294400.sst): second pass time = 1.34's, 80.626'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.791'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3407.102'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 284.888'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32234.432'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.626'MB/sec zip pipeline throughput = 80.626'MB/sec entries = 932047 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.00 avg-zval = 33.06 usrkeys = 932047 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.00 avg-zval = 33.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932047 bytes = 3262176 } seq expand size = 6524329 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5135 value = 3.5087 dict = 1.00 all = 3.6340 } Zip/UnZip{ index = 0.0951 value = 0.2850 dict = 1.00 all = 0.2752 } ---------------------------- total value len = 2252.587596 GB avg = 0.200 KB (by entry num) total key len = 307.168793 GB avg = 0.027 KB total ukey len = 237.558892 GB avg = 0.015 KB total ukey num = 15.748869391 Billion total entry num = 11.290183339 Billion write speed all = 33.611011842 MB/sec (with seq num) write speed all = 32.425041216 MB/sec (without seq num) 2024/11/24-06:04:30.526866 139957609621056 [compaction_job.cc:1948] [default] [JOB 42573] Generated table #294400: 932047 keys, 36934318 bytes, temperature: kWarm 2024/11/24-06:04:30.526940 139957609621056 EVENT_LOG_v1 {"time_micros": 1732399470526904, "cf_name": "default", "job": 42573, "event": "table_file_creation", "file_number": 294400, "file_size": 36934318, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679902, "largest_seqno": 4470947778, "table_properties": {"data_size": 30815344, "index_size": 1773056, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26097316, "raw_average_key_size": 28, "raw_value_size": 108120442, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 932047, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399467, "oldest_key_time": 0, "file_creation_time": 1732399467, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294400, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:04:30.569156 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294401.sst) /dev/shm/ToplingTemp/Topling-285412-5GU41I 2024/11/24-06:04:31.569215 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294401.sst): first pass time = 1.00's, 134.215'MB/sec 2024/11/24-06:04:32.865501 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294401.sst): old prealloc_size = 147639500, real_size = 31408112 2024/11/24-06:04:32.866098 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294401.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 32.00 2024/11/24-06:04:32.868075 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294401.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:04:33.680110 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294401.sst): second pass time = 1.30's, 83.735'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.421'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3058.177'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 274.833'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 33195.961'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.735'MB/sec zip pipeline throughput = 83.735'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.00 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4455 value = 3.6875 dict = 1.00 all = 3.7947 } Zip/UnZip{ index = 0.0957 value = 0.2712 dict = 1.00 all = 0.2635 } ---------------------------- total value len = 2252.702509 GB avg = 0.200 KB (by entry num) total key len = 307.194533 GB avg = 0.027 KB total ukey len = 237.577278 GB avg = 0.015 KB total ukey num = 15.749788691 Billion total entry num = 11.291102639 Billion write speed all = 33.611446768 MB/sec (with seq num) write speed all = 32.425429396 MB/sec (without seq num) 2024/11/24-06:04:33.723971 139957609621056 [compaction_job.cc:1948] [default] [JOB 42573] Generated table #294401: 919300 keys, 35369970 bytes, temperature: kWarm 2024/11/24-06:04:33.724065 139957609621056 EVENT_LOG_v1 {"time_micros": 1732399473724027, "cf_name": "default", "job": 42573, "event": "table_file_creation", "file_number": 294401, "file_size": 35369970, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4312213990, "largest_seqno": 4422977642, "table_properties": {"data_size": 29417888, "index_size": 1760192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399470, "oldest_key_time": 0, "file_creation_time": 1732399470, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294401, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:04:33.762346 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294402.sst) /dev/shm/ToplingTemp/Topling-285412-mvxO7H 2024/11/24-06:04:34.774175 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294402.sst): first pass time = 1.01's, 132.655'MB/sec 2024/11/24-06:04:36.090380 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294402.sst): old prealloc_size = 147639500, real_size = 30865856 2024/11/24-06:04:36.090965 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294402.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932139 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0289 GB avg-val = 115.99 avg-zval = 30.96 2024/11/24-06:04:36.093077 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294402.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:04:36.922874 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294402.sst): second pass time = 1.32's, 82.191'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.018'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3146.576'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.332'MB/sec write SST data time = 0.13's, 268.768'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 25138.440'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.191'MB/sec zip pipeline throughput = 82.191'MB/sec entries = 932139 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.99 avg-zval = 30.96 usrkeys = 932139 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.99 avg-zval = 30.96 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932139 bytes = 3262496 } seq expand size = 6524973 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.5149 value = 3.7463 dict = 1.00 all = 3.8371 } Zip/UnZip{ index = 0.0951 value = 0.2669 dict = 1.00 all = 0.2606 } ---------------------------- total value len = 2252.817151 GB avg = 0.200 KB (by entry num) total key len = 307.220633 GB avg = 0.027 KB total ukey len = 237.595921 GB avg = 0.015 KB total ukey num = 15.750720830 Billion total entry num = 11.292034778 Billion write speed all = 33.611863611 MB/sec (with seq num) write speed all = 32.425798826 MB/sec (without seq num) 2024/11/24-06:04:36.966227 139957609621056 [compaction_job.cc:1948] [default] [JOB 42573] Generated table #294402: 932139 keys, 34979198 bytes, temperature: kWarm 2024/11/24-06:04:36.966298 139957609621056 EVENT_LOG_v1 {"time_micros": 1732399476966262, "cf_name": "default", "job": 42573, "event": "table_file_creation", "file_number": 294402, "file_size": 34979198, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679921, "largest_seqno": 4470947798, "table_properties": {"data_size": 28859632, "index_size": 1772992, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26099892, "raw_average_key_size": 28, "raw_value_size": 108117946, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932139, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399473, "oldest_key_time": 0, "file_creation_time": 1732399473, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294402, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:04:37.009157 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294403.sst) /dev/shm/ToplingTemp/Topling-285412-wKkCdH 2024/11/24-06:04:38.021152 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294403.sst): first pass time = 1.01's, 132.632'MB/sec 2024/11/24-06:04:39.324907 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294403.sst): old prealloc_size = 147639500, real_size = 31452672 2024/11/24-06:04:39.325517 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294403.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.05 2024/11/24-06:04:39.327506 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294403.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:04:40.146959 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294403.sst): second pass time = 1.30's, 83.282'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.082'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2994.456'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.13's, 273.393'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 31414.054'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.282'MB/sec zip pipeline throughput = 83.282'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.05 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4455 value = 3.6819 dict = 1.00 all = 3.7899 } Zip/UnZip{ index = 0.0957 value = 0.2716 dict = 1.00 all = 0.2639 } ---------------------------- total value len = 2252.932064 GB avg = 0.199 KB (by entry num) total key len = 307.246374 GB avg = 0.027 KB total ukey len = 237.614307 GB avg = 0.015 KB total ukey num = 15.751640130 Billion total entry num = 11.292954078 Billion write speed all = 33.612287485 MB/sec (with seq num) write speed all = 32.426176350 MB/sec (without seq num) 2024/11/24-06:04:40.191074 139957609621056 [compaction_job.cc:1948] [default] [JOB 42573] Generated table #294403: 919300 keys, 35414514 bytes, temperature: kWarm 2024/11/24-06:04:40.191147 139957609621056 EVENT_LOG_v1 {"time_micros": 1732399480191109, "cf_name": "default", "job": 42573, "event": "table_file_creation", "file_number": 294403, "file_size": 35414514, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4273749156, "largest_seqno": 4382206945, "table_properties": {"data_size": 29462448, "index_size": 1760192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399476, "oldest_key_time": 0, "file_creation_time": 1732399476, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294403, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:04:40.226147 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294404.sst) /dev/shm/ToplingTemp/Topling-285412-NRppzI 2024/11/24-06:04:41.130770 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294404.sst): first pass time = 0.90's, 148.375'MB/sec 2024/11/24-06:04:42.428539 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294404.sst): old prealloc_size = 147639500, real_size = 30870064 2024/11/24-06:04:42.429101 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294404.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924830 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0289 GB avg-val = 117.13 avg-zval = 31.21 2024/11/24-06:04:42.429890 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294404.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:04:43.247765 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294404.sst): second pass time = 1.30's, 83.520'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.536'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3290.402'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 269.501'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31010.783'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.520'MB/sec zip pipeline throughput = 83.520'MB/sec entries = 924830 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.13 avg-zval = 31.21 usrkeys = 924830 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.13 avg-zval = 31.21 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924830 bytes = 3121312 } seq expand size = 6473810 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0348 GB } UnZip/Zip{ index = 10.4437 value = 3.7524 dict = 1.00 all = 3.8518 } Zip/UnZip{ index = 0.0958 value = 0.2665 dict = 1.00 all = 0.2596 } ---------------------------- total value len = 2253.046860 GB avg = 0.199 KB (by entry num) total key len = 307.272269 GB avg = 0.027 KB total ukey len = 237.632803 GB avg = 0.015 KB total ukey num = 15.752564960 Billion total entry num = 11.293878908 Billion write speed all = 33.612766227 MB/sec (with seq num) write speed all = 32.426606245 MB/sec (without seq num) 2024/11/24-06:04:43.291739 139957609621056 [compaction_job.cc:1948] [default] [JOB 42573] Generated table #294404: 924830 keys, 34845522 bytes, temperature: kWarm 2024/11/24-06:04:43.291813 139957609621056 EVENT_LOG_v1 {"time_micros": 1732399483291775, "cf_name": "default", "job": 42573, "event": "table_file_creation", "file_number": 294404, "file_size": 34845522, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4382206946, "largest_seqno": 4470947814, "table_properties": {"data_size": 28867616, "index_size": 1771072, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25895240, "raw_average_key_size": 28, "raw_value_size": 108322488, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924830, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399480, "oldest_key_time": 0, "file_creation_time": 1732399480, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294404, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:04:43.331435 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294405.sst) /dev/shm/ToplingTemp/Topling-285412-lD8nwJ 2024/11/24-06:04:44.339380 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294405.sst): first pass time = 1.01's, 133.165'MB/sec 2024/11/24-06:04:45.672303 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294405.sst): old prealloc_size = 147639500, real_size = 32929008 2024/11/24-06:04:45.672863 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294405.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926629 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 116.85 avg-zval = 33.37 2024/11/24-06:04:45.674930 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294405.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:04:46.526205 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294405.sst): second pass time = 1.33's, 81.274'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.363'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3316.455'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 285.175'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31345.772'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.274'MB/sec zip pipeline throughput = 81.274'MB/sec entries = 926629 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.85 avg-zval = 33.37 usrkeys = 926629 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.85 avg-zval = 33.37 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926629 bytes = 3243216 } seq expand size = 6486403 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.4459 value = 3.5013 dict = 1.00 all = 3.6249 } Zip/UnZip{ index = 0.0957 value = 0.2856 dict = 1.00 all = 0.2759 } ---------------------------- total value len = 2253.161619 GB avg = 0.199 KB (by entry num) total key len = 307.298214 GB avg = 0.027 KB total ukey len = 237.651336 GB avg = 0.015 KB total ukey num = 15.753491589 Billion total entry num = 11.294805537 Billion write speed all = 33.613166710 MB/sec (with seq num) write speed all = 32.426960462 MB/sec (without seq num) 2024/11/24-06:04:46.571638 139957609621056 [compaction_job.cc:1948] [default] [JOB 42573] Generated table #294405: 926629 keys, 37026686 bytes, temperature: kWarm 2024/11/24-06:04:46.571710 139957609621056 EVENT_LOG_v1 {"time_micros": 1732399486571675, "cf_name": "default", "job": 42573, "event": "table_file_creation", "file_number": 294405, "file_size": 37026686, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679941, "largest_seqno": 4470945927, "table_properties": {"data_size": 30923040, "index_size": 1774144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25945612, "raw_average_key_size": 28, "raw_value_size": 108272182, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926629, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399483, "oldest_key_time": 0, "file_creation_time": 1732399483, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294405, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:04:46.610518 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294406.sst) /dev/shm/ToplingTemp/Topling-285412-vyC7nG 2024/11/24-06:04:47.601152 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294406.sst): first pass time = 0.99's, 135.492'MB/sec 2024/11/24-06:04:48.924611 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294406.sst): old prealloc_size = 147639500, real_size = 31219696 2024/11/24-06:04:48.925183 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294406.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0292 GB avg-val = 118.00 avg-zval = 31.80 2024/11/24-06:04:48.927109 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294406.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:04:49.742287 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294406.sst): second pass time = 1.32's, 82.012'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.028'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3188.672'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 269.900'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 34314.042'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.012'MB/sec zip pipeline throughput = 82.012'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.80 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.80 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.09 MB all = 0.0352 GB } UnZip/Zip{ index = 10.4458 value = 3.7112 dict = 1.00 all = 3.8150 } Zip/UnZip{ index = 0.0957 value = 0.2695 dict = 1.00 all = 0.2621 } ---------------------------- total value len = 2253.276531 GB avg = 0.199 KB (by entry num) total key len = 307.323955 GB avg = 0.027 KB total ukey len = 237.669722 GB avg = 0.015 KB total ukey num = 15.754410889 Billion total entry num = 11.295724837 Billion write speed all = 33.613594008 MB/sec (with seq num) write speed all = 32.427341296 MB/sec (without seq num) 2024/11/24-06:04:49.784789 139957609621056 [compaction_job.cc:1948] [default] [JOB 42573] Generated table #294406: 919300 keys, 35181554 bytes, temperature: kWarm 2024/11/24-06:04:49.784862 139957609621056 EVENT_LOG_v1 {"time_micros": 1732399489784826, "cf_name": "default", "job": 42573, "event": "table_file_creation", "file_number": 294406, "file_size": 35181554, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4342088854, "largest_seqno": 4454019919, "table_properties": {"data_size": 29229536, "index_size": 1760128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399486, "oldest_key_time": 0, "file_creation_time": 1732399486, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294406, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:04:49.820817 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294407.sst) /dev/shm/ToplingTemp/Topling-285412-NgCwlG 2024/11/24-06:04:50.638413 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294407.sst): first pass time = 0.82's, 164.170'MB/sec 2024/11/24-06:04:51.979763 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294407.sst): old prealloc_size = 147639500, real_size = 32454592 2024/11/24-06:04:51.980366 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294407.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932238 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0304 GB avg-val = 115.97 avg-zval = 32.66 2024/11/24-06:04:51.981143 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294407.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:04:52.838586 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294407.sst): second pass time = 1.34's, 80.647'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.581'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3064.881'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.13's, 283.585'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 25903.698'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.647'MB/sec zip pipeline throughput = 80.647'MB/sec entries = 932238 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.97 avg-zval = 32.66 usrkeys = 932238 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.97 avg-zval = 32.66 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932238 bytes = 3262848 } seq expand size = 6525666 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5145 value = 3.5508 dict = 1.00 all = 3.6703 } Zip/UnZip{ index = 0.0951 value = 0.2816 dict = 1.00 all = 0.2725 } ---------------------------- total value len = 2253.391172 GB avg = 0.199 KB (by entry num) total key len = 307.350057 GB avg = 0.027 KB total ukey len = 237.688367 GB avg = 0.015 KB total ukey num = 15.755343127 Billion total entry num = 11.296657075 Billion write speed all = 33.614075307 MB/sec (with seq num) write speed all = 32.427772912 MB/sec (without seq num) 2024/11/24-06:04:52.883418 139957609621056 [compaction_job.cc:1948] [default] [JOB 42573] Generated table #294407: 932238 keys, 36568342 bytes, temperature: kWarm 2024/11/24-06:04:52.883492 139957609621056 EVENT_LOG_v1 {"time_micros": 1732399492883454, "cf_name": "default", "job": 42573, "event": "table_file_creation", "file_number": 294407, "file_size": 36568342, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679961, "largest_seqno": 4470947834, "table_properties": {"data_size": 30448112, "index_size": 1773248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26102664, "raw_average_key_size": 28, "raw_value_size": 108115140, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932238, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399489, "oldest_key_time": 0, "file_creation_time": 1732399489, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294407, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:04:52.929445 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294408.sst) /dev/shm/ToplingTemp/Topling-285412-gdfXoH 2024/11/24-06:04:53.924593 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294408.sst): first pass time = 1.00's, 134.877'MB/sec 2024/11/24-06:04:55.215613 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294408.sst): old prealloc_size = 147639500, real_size = 31423920 2024/11/24-06:04:55.216197 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294408.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 32.00 2024/11/24-06:04:55.217109 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294408.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:04:56.032711 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294408.sst): second pass time = 1.29's, 84.076'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.308'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3143.837'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 274.790'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 31505.270'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.076'MB/sec zip pipeline throughput = 84.076'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 32.00 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 32.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0354 GB } UnZip/Zip{ index = 10.3764 value = 3.6870 dict = 1.00 all = 3.7930 } Zip/UnZip{ index = 0.0964 value = 0.2712 dict = 1.00 all = 0.2636 } ---------------------------- total value len = 2253.506085 GB avg = 0.199 KB (by entry num) total key len = 307.375798 GB avg = 0.027 KB total ukey len = 237.706753 GB avg = 0.015 KB total ukey num = 15.756262427 Billion total entry num = 11.297576375 Billion write speed all = 33.614512213 MB/sec (with seq num) write speed all = 32.428163021 MB/sec (without seq num) 2024/11/24-06:04:56.075645 139957609621056 [compaction_job.cc:1948] [default] [JOB 42573] Generated table #294408: 919300 keys, 35385778 bytes, temperature: kWarm 2024/11/24-06:04:56.075716 139957609621056 EVENT_LOG_v1 {"time_micros": 1732399496075681, "cf_name": "default", "job": 42573, "event": "table_file_creation", "file_number": 294408, "file_size": 35385778, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4302514552, "largest_seqno": 4413000005, "table_properties": {"data_size": 29421984, "index_size": 1771904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399492, "oldest_key_time": 0, "file_creation_time": 1732399492, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294408, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:04:56.112471 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294409.sst) /dev/shm/ToplingTemp/Topling-285412-VNjkiK 2024/11/24-06:04:57.064104 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294409.sst): first pass time = 0.95's, 141.046'MB/sec 2024/11/24-06:04:58.359223 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294409.sst): old prealloc_size = 147639500, real_size = 30126656 2024/11/24-06:04:58.359818 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294409.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931898 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0281 GB avg-val = 116.03 avg-zval = 30.16 2024/11/24-06:04:58.361794 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294409.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:04:59.185573 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294409.sst): second pass time = 1.29's, 83.535'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 41.151'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3117.318'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.334'MB/sec write SST data time = 0.13's, 263.210'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32991.247'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.535'MB/sec zip pipeline throughput = 83.535'MB/sec entries = 931898 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.03 avg-zval = 30.16 usrkeys = 931898 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.03 avg-zval = 30.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931898 bytes = 3028688 } seq expand size = 6523286 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0281 GB dict = 1.08 MB all = 0.0340 GB } UnZip/Zip{ index = 10.4450 value = 3.8466 dict = 1.00 all = 3.9471 } Zip/UnZip{ index = 0.0957 value = 0.2600 dict = 1.00 all = 0.2534 } ---------------------------- total value len = 2253.620733 GB avg = 0.199 KB (by entry num) total key len = 307.401891 GB avg = 0.027 KB total ukey len = 237.725391 GB avg = 0.015 KB total ukey num = 15.757194325 Billion total entry num = 11.298508273 Billion write speed all = 33.614968451 MB/sec (with seq num) write speed all = 32.428570500 MB/sec (without seq num) 2024/11/24-06:04:59.228723 139957609621056 [compaction_job.cc:1948] [default] [JOB 42573] Generated table #294409: 931898 keys, 34004308 bytes, temperature: kWarm 2024/11/24-06:04:59.228795 139957609621056 EVENT_LOG_v1 {"time_micros": 1732399499228759, "cf_name": "default", "job": 42573, "event": "table_file_creation", "file_number": 294409, "file_size": 34004308, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4413000006, "largest_seqno": 4470947854, "table_properties": {"data_size": 28109104, "index_size": 1784384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26093144, "raw_average_key_size": 28, "raw_value_size": 108124720, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931898, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399496, "oldest_key_time": 0, "file_creation_time": 1732399496, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294409, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:04:59.265687 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294410.sst) /dev/shm/ToplingTemp/Topling-285412-1wkBIK 2024/11/24-06:05:00.268390 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294410.sst): first pass time = 1.00's, 133.860'MB/sec 2024/11/24-06:05:01.563344 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294410.sst): old prealloc_size = 147639500, real_size = 31553840 2024/11/24-06:05:01.563878 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294410.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919702 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1085 GB zip-val = 0.0296 GB avg-val = 117.94 avg-zval = 32.16 2024/11/24-06:05:01.564704 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294410.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:05:02.382486 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294410.sst): second pass time = 1.29's, 83.841'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.503'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3528.952'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.13's, 277.447'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 30533.956'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.841'MB/sec zip pipeline throughput = 83.841'MB/sec entries = 919702 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.94 avg-zval = 32.16 usrkeys = 919702 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.94 avg-zval = 32.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919702 bytes = 3218976 } seq expand size = 6437914 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0296 GB dict = 1.08 MB all = 0.0356 GB } UnZip/Zip{ index = 10.5150 value = 3.6676 dict = 1.00 all = 3.7666 } Zip/UnZip{ index = 0.0951 value = 0.2727 dict = 1.00 all = 0.2655 } ---------------------------- total value len = 2253.735637 GB avg = 0.199 KB (by entry num) total key len = 307.427643 GB avg = 0.027 KB total ukey len = 237.743785 GB avg = 0.015 KB total ukey num = 15.758114027 Billion total entry num = 11.299427975 Billion write speed all = 33.615404094 MB/sec (with seq num) write speed all = 32.428959355 MB/sec (without seq num) 2024/11/24-06:05:02.427184 139957609621056 [compaction_job.cc:1948] [default] [JOB 42573] Generated table #294410: 919702 keys, 35633590 bytes, temperature: kWarm 2024/11/24-06:05:02.427258 139957609621056 EVENT_LOG_v1 {"time_micros": 1732399502427220, "cf_name": "default", "job": 42573, "event": "table_file_creation", "file_number": 294410, "file_size": 35633590, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679976, "largest_seqno": 4470942661, "table_properties": {"data_size": 29574432, "index_size": 1749312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25751656, "raw_average_key_size": 28, "raw_value_size": 108466192, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 919702, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399499, "oldest_key_time": 0, "file_creation_time": 1732399499, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294410, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:05:02.465663 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294411.sst) /dev/shm/ToplingTemp/Topling-285412-xYk7JG 2024/11/24-06:05:03.273318 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294411.sst): first pass time = 0.81's, 166.193'MB/sec 2024/11/24-06:05:04.582509 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294411.sst): old prealloc_size = 147639500, real_size = 31050944 2024/11/24-06:05:04.583125 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294411.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922387 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0291 GB avg-val = 117.51 avg-zval = 31.50 2024/11/24-06:05:04.585097 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294411.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:05:05.397198 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294411.sst): second pass time = 1.31's, 82.842'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 42.988'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2995.317'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.337'MB/sec write SST data time = 0.13's, 270.977'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34307.819'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.842'MB/sec zip pipeline throughput = 82.842'MB/sec entries = 922387 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.51 avg-zval = 31.50 usrkeys = 922387 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.51 avg-zval = 31.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922387 bytes = 3113072 } seq expand size = 6456709 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.4433 value = 3.7307 dict = 1.00 all = 3.8325 } Zip/UnZip{ index = 0.0958 value = 0.2680 dict = 1.00 all = 0.2609 } ---------------------------- total value len = 2253.850484 GB avg = 0.199 KB (by entry num) total key len = 307.453470 GB avg = 0.027 KB total ukey len = 237.762232 GB avg = 0.015 KB total ukey num = 15.759036414 Billion total entry num = 11.300350362 Billion write speed all = 33.615920329 MB/sec (with seq num) write speed all = 32.429425687 MB/sec (without seq num) 2024/11/24-06:05:05.441422 139957609621056 [compaction_job.cc:1948] [default] [JOB 42573] Generated table #294411: 922387 keys, 35020698 bytes, temperature: kWarm 2024/11/24-06:05:05.441491 139957609621056 EVENT_LOG_v1 {"time_micros": 1732399505441457, "cf_name": "default", "job": 42573, "event": "table_file_creation", "file_number": 294411, "file_size": 35020698, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4371735807, "largest_seqno": 4470947873, "table_properties": {"data_size": 29053680, "index_size": 1766464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25826836, "raw_average_key_size": 28, "raw_value_size": 108390898, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922387, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399502, "oldest_key_time": 0, "file_creation_time": 1732399502, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294411, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:05:05.477719 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294412.sst) /dev/shm/ToplingTemp/Topling-285412-Hdna9H 2024/11/24-06:05:05.682867 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294412.sst): first pass time = 0.21's, 242.187'MB/sec 2024/11/24-06:05:06.155571 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294412.sst): old prealloc_size = 147639500, real_size = 12124368 2024/11/24-06:05:06.155835 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294412.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 349822 prefix = 4 raw-key = 0.0056 GB zip-key = 0.0007 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0399 GB zip-val = 0.0114 GB avg-val = 114.00 avg-zval = 32.49 2024/11/24-06:05:06.156192 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294412.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0008 GB, waited 0.000 sec, Key+Value = 0.050 GB 2024/11/24-06:05:06.473335 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00/294412.sst): second pass time = 0.47's, 84.495'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.32's, 42.742'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2827.298'MB/sec (index lex order gen) rebuild zvType time = 0.27's, 0.325'MB/sec write SST data time = 0.05's, 281.569'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.405'MB, throughput = 11299.475'MB/sec zip my value time = 0.47's, unzip length = 0.040'GB zip my value throughput = 84.495'MB/sec zip pipeline throughput = 84.495'MB/sec entries = 349822 avg-key = 28.00 avg-zkey = 1.92 avg-val = 114.00 avg-zval = 32.49 usrkeys = 349822 avg-key = 20.00 avg-zkey = 1.92 avg-val = 121.00 avg-zval = 32.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 349822 bytes = 1136944 } seq expand size = 2448754 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0070 GB value = 0.0399 GB dict = 0.40 MB all = 0.0497 GB } __ZipSize{ index = 0.0007 GB value = 0.0114 GB dict = 0.40 MB all = 0.0136 GB } UnZip/Zip{ index = 10.4312 value = 3.5087 dict = 1.00 all = 3.6578 } Zip/UnZip{ index = 0.0959 value = 0.2850 dict = 1.00 all = 0.2734 } ---------------------------- total value len = 2253.892813 GB avg = 0.199 KB (by entry num) total key len = 307.463265 GB avg = 0.027 KB total ukey len = 237.769229 GB avg = 0.015 KB total ukey num = 15.759386236 Billion total entry num = 11.300700184 Billion write speed all = 33.616129638 MB/sec (with seq num) write speed all = 32.429615023 MB/sec (without seq num) 2024/11/24-06:05:06.494510 139957609621056 [compaction_job.cc:1948] [default] [JOB 42573] Generated table #294412: 349822 keys, 13580639 bytes, temperature: kWarm 2024/11/24-06:05:06.494570 139957609621056 EVENT_LOG_v1 {"time_micros": 1732399506494539, "cf_name": "default", "job": 42573, "event": "table_file_creation", "file_number": 294412, "file_size": 13580639, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4432877723, "largest_seqno": 4470941900, "table_properties": {"data_size": 11366016, "index_size": 670720, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 9795016, "raw_average_key_size": 28, "raw_value_size": 39879708, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 349822, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399505, "oldest_key_time": 0, "file_creation_time": 1732399505, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294412, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:05:06.512582 139957609621056 [compaction_job.cc:703] job-42573: subcompact[0], size: 0.792857 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42573/att-00 [294390,294391,294392,294393,294394,294395,294396,294397,294398,294399,294400,294401,294402,294403,294404,294405,294406,294407,294408,294409,294410,294411,294412] 2024/11/24-06:05:06.644219 139957609621056 (Original Log Time 2024/11/24-06:03:52.430868) EVENT_LOG_v1 {"time_micros": 1732399432430840, "job": 42573, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [294347], "files_L3": [294387, 294290, 294291, 294292, 294293, 294294, 294295, 294296, 294297, 294298, 294299, 294300, 294301, 294302, 294303, 294304, 294305, 294306, 294307, 294264, 294265, 294266, 294267], "score": 0.948728, "input_data_size": 818251683, "oldest_snapshot_seqno": 4471362747} 2024/11/24-06:05:06.644222 139957609621056 (Original Log Time 2024/11/24-06:05:06.642543) [compaction_job.cc:2022] [default] [JOB 42573] Compacted 1@2 + 23@3 files to L3 => 792857109 bytes 2024/11/24-06:05:06.644223 139957609621056 (Original Log Time 2024/11/24-06:05:06.644141) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 23 0 0 0] max score 0.03, MB/sec: 11.0 rd, 10.7 wr, level 3, files in(1, 23) out(23 +0 blob) MB in(26.6, 753.7 +0.0 blob) out(756.1 +0.0 blob), read-write-amplify(57.7) write-amplify(28.4) OK, records in: 20829665, records dropped: 104592 output_compression: Snappy 2024/11/24-06:05:06.644227 139957609621056 (Original Log Time 2024/11/24-06:05:06.644191) EVENT_LOG_v1 {"time_micros": 1732399506644166, "cf": "default", "job": 42573, "event": "compaction_finished", "compaction_time_micros": 74081721, "compaction_time_cpu_micros": 31706923, "output_level": 3, "num_output_files": 23, "total_output_size": 792857109, "num_input_records": 20829665, "num_output_records": 20725073, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 560910293, "file_range_sync_nanos": 0, "file_fsync_nanos": 16106, "file_prepare_write_nanos": 62460, "lsm_state": [0, 0, 0, 23, 0, 0, 0]} 2024/11/24-06:05:06.766187 139957609621056 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732399432,"executesMs":74335,"compactionJobId":42573,"attempt":0,"compactionInputRawBytes":3017318266,"compactionInputZipBytes":818251683,"compactionOutputRawBytes":0,"compactionOutputZipBytes":792857109,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-41210bab40f54d180a5e5ba02c30cebb-cb850de32db74f24-00"}}, response = 2024/11/24-06:05:06.766285 139957609621056 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-42573/att-00: olev 3, work 74.336 s, result 7.774 ms, install 1.700 ms, input{raw 2.810 GiB zip 780.346 MiB}