2024/11/23-15:01:51.314134 139957181593152 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00: fork to child time = 0.006343 sec 2024/11/23-15:01:51.314764 139957181593152 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00 2024/11/23-15:01:51.321932 139957181593152 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00 2024/11/23-15:01:51.322830 139957181593152 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-15:01:51.323084 139957181593152 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-15:01:51.378356 139957181593152 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28676/att-00/MANIFEST-206347 2024/11/23-15:01:51.401056 139957181593152 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28676/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 265179, last_sequence is 4049168731, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 265050 2024/11/23-15:01:51.401063 139957181593152 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-15:01:51.401192 139957181593152 [version_set.cc:5721] Creating manifest 265179 2024/11/23-15:01:51.435352 139957181593152 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-15:01:51.457381 139957181593152 [compaction_job.cc:2437] [default] [JOB 28676] Compacting 1@2 + 24@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-15:01:51.457396 139957181593152 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 21431 Base level 2, inputs: [265135(27MB)], [265077(39MB) 265078(38MB) 265079(33MB) 265080(35MB) 265081(34MB) 265082(35MB) 265083(34MB) 265084(36MB) 265085(33MB) 265086(33MB) 265087(33MB) 265088(34MB) 265089(38MB) 265090(36MB) 265091(35MB) 265092(34MB) 265093(35MB) 265094(35MB) 265095(36MB) 265096(34MB) 265097(34MB) 265098(36MB) 265099(37MB) 265147(39MB)] 2024/11/23-15:01:51.464113 139957181593152 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-15:01:51.464136 139957181593152 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-15:01:51.464152 139957181593152 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-15:01:51.464158 139957181593152 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-15:01:51.464172 139957181593152 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-15:01:52.088723 139957181593152 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265180.sst) /dev/shm/ToplingTemp/Topling-85660-GWldIp 2024/11/23-15:01:52.693973 139957181593152 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265180.sst): first pass time = 0.61's, 221.771'MB/sec 2024/11/23-15:01:54.184785 139957181593152 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265180.sst): old prealloc_size = 147639500, real_size = 37353488 2024/11/23-15:01:54.185369 139957181593152 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265180.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926496 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0354 GB avg-val = 116.87 avg-zval = 38.16 2024/11/23-15:01:54.186138 139957181593152 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265180.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:01:55.173837 139957181593152 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265180.sst): second pass time = 1.49's, 72.700'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.99's, 41.676'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3133.901'MB/sec (index lex order gen) rebuild zvType time = 0.86's, 0.269'MB/sec write SST data time = 0.13's, 324.334'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 42870.531'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.700'MB/sec zip pipeline throughput = 72.700'MB/sec entries = 926496 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.87 avg-zval = 38.16 usrkeys = 926496 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.87 avg-zval = 38.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926496 bytes = 3011120 } seq expand size = 6485472 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0354 GB dict = 1.08 MB all = 0.0412 GB } UnZip/Zip{ index = 10.5142 value = 3.0622 dict = 1.00 all = 3.2562 } Zip/UnZip{ index = 0.0951 value = 0.3266 dict = 1.00 all = 0.3071 } ---------------------------- total value len = 8.963381 GB avg = 0.168 KB (by entry num) total key len = 1.466763 GB avg = 0.028 KB total ukey len = 1.016408 GB avg = 0.015 KB total ukey num = 0.066476931 Billion total entry num = 0.053284660 Billion write speed all = 0.474033700 MB/sec (with seq num) write speed all = 0.454660066 MB/sec (without seq num) 2024/11/23-15:01:55.241469 139957181593152 [compaction_job.cc:1948] [default] [JOB 28676] Generated table #265180: 926496 keys, 41219234 bytes, temperature: kWarm 2024/11/23-15:01:55.241592 139957181593152 EVENT_LOG_v1 {"time_micros": 1732345315241514, "cf_name": "default", "job": 28676, "event": "table_file_creation", "file_number": 265180, "file_size": 41219234, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4012777888, "largest_seqno": 4048637136, "table_properties": {"data_size": 35359296, "index_size": 1762368, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25941888, "raw_average_key_size": 28, "raw_value_size": 108275920, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926496, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345312, "oldest_key_time": 0, "file_creation_time": 1732345312, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265180, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:01:55.281053 139957181593152 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265181.sst) /dev/shm/ToplingTemp/Topling-85660-5Fs1wn 2024/11/23-15:01:56.207861 139957181593152 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265181.sst): first pass time = 0.93's, 144.824'MB/sec 2024/11/23-15:01:57.602265 139957181593152 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265181.sst): old prealloc_size = 147639500, real_size = 36477248 2024/11/23-15:01:57.602819 139957181593152 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265181.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926331 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.89 avg-zval = 37.23 2024/11/23-15:01:57.603604 139957181593152 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265181.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:01:58.424058 139957181593152 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265181.sst): second pass time = 1.39's, 77.701'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 49.519'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3327.623'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 319.106'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33459.163'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 77.701'MB/sec zip pipeline throughput = 77.701'MB/sec entries = 926331 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.89 avg-zval = 37.23 usrkeys = 926331 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.89 avg-zval = 37.23 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926331 bytes = 3357968 } seq expand size = 6484317 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0407 GB } UnZip/Zip{ index = 10.5135 value = 3.1401 dict = 1.00 all = 3.2983 } Zip/UnZip{ index = 0.0951 value = 0.3185 dict = 1.00 all = 0.3032 } ---------------------------- total value len = 9.078146 GB avg = 0.167 KB (by entry num) total key len = 1.492700 GB avg = 0.028 KB total ukey len = 1.034935 GB avg = 0.015 KB total ukey num = 0.067403262 Billion total entry num = 0.054210991 Billion write speed all = 0.480357433 MB/sec (with seq num) write speed all = 0.460649908 MB/sec (without seq num) 2024/11/23-15:01:58.473017 139957181593152 [compaction_job.cc:1948] [default] [JOB 28676] Generated table #265181: 926331 keys, 40693624 bytes, temperature: kWarm 2024/11/23-15:01:58.473086 139957181593152 EVENT_LOG_v1 {"time_micros": 1732345318473051, "cf_name": "default", "job": 28676, "event": "table_file_creation", "file_number": 265181, "file_size": 40693624, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 4048635756, "table_properties": {"data_size": 34483312, "index_size": 1762176, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25937268, "raw_average_key_size": 28, "raw_value_size": 108280534, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926331, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345315, "oldest_key_time": 0, "file_creation_time": 1732345315, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265181, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:01:58.510411 139957181593152 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265182.sst) /dev/shm/ToplingTemp/Topling-85660-bkBhbq 2024/11/23-15:01:59.346196 139957181593152 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265182.sst): first pass time = 0.84's, 160.596'MB/sec 2024/11/23-15:02:00.668495 139957181593152 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265182.sst): old prealloc_size = 147639500, real_size = 31618928 2024/11/23-15:02:00.669086 139957181593152 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265182.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931070 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0296 GB avg-val = 116.15 avg-zval = 31.80 2024/11/23-15:02:00.669883 139957181593152 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265182.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:02:01.506949 139957181593152 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265182.sst): second pass time = 1.32's, 81.847'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.753'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3136.442'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.328'MB/sec write SST data time = 0.13's, 281.486'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 27902.290'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.847'MB/sec zip pipeline throughput = 81.847'MB/sec entries = 931070 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.15 avg-zval = 31.80 usrkeys = 931070 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.15 avg-zval = 31.80 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931070 bytes = 3375136 } seq expand size = 6517490 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.08 MB all = 0.0358 GB } UnZip/Zip{ index = 10.4466 value = 3.6532 dict = 1.00 all = 3.7444 } Zip/UnZip{ index = 0.0957 value = 0.2737 dict = 1.00 all = 0.2671 } ---------------------------- total value len = 9.192811 GB avg = 0.167 KB (by entry num) total key len = 1.518770 GB avg = 0.028 KB total ukey len = 1.053556 GB avg = 0.015 KB total ukey num = 0.068334332 Billion total entry num = 0.055142061 Billion write speed all = 0.486684505 MB/sec (with seq num) write speed all = 0.466641312 MB/sec (without seq num) 2024/11/23-15:02:01.551077 139957181593152 [compaction_job.cc:1948] [default] [JOB 28676] Generated table #265182: 931070 keys, 35845404 bytes, temperature: kWarm 2024/11/23-15:02:01.551151 139957181593152 EVENT_LOG_v1 {"time_micros": 1732345321551113, "cf_name": "default", "job": 28676, "event": "table_file_creation", "file_number": 265182, "file_size": 35845404, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3621075956, "largest_seqno": 4012776133, "table_properties": {"data_size": 29603424, "index_size": 1782528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26069960, "raw_average_key_size": 28, "raw_value_size": 108147808, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931070, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345318, "oldest_key_time": 0, "file_creation_time": 1732345318, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265182, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:02:01.588442 139957181593152 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265183.sst) /dev/shm/ToplingTemp/Topling-85660-uTeMOo 2024/11/23-15:02:02.612330 139957181593152 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265183.sst): first pass time = 1.02's, 131.092'MB/sec 2024/11/23-15:02:03.957131 139957181593152 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265183.sst): old prealloc_size = 147639500, real_size = 33277392 2024/11/23-15:02:03.957689 139957181593152 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265183.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930877 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0313 GB avg-val = 116.18 avg-zval = 33.58 2024/11/23-15:02:03.958504 139957181593152 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265183.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:02:04.797597 139957181593152 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265183.sst): second pass time = 1.34's, 80.475'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.762'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3336.116'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.325'MB/sec write SST data time = 0.12's, 302.758'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 37102.739'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.475'MB/sec zip pipeline throughput = 80.475'MB/sec entries = 930877 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.18 avg-zval = 33.58 usrkeys = 930877 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.18 avg-zval = 33.58 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930877 bytes = 3490800 } seq expand size = 6516139 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.4452 value = 3.4596 dict = 1.00 all = 3.5678 } Zip/UnZip{ index = 0.0957 value = 0.2891 dict = 1.00 all = 0.2803 } ---------------------------- total value len = 9.307481 GB avg = 0.166 KB (by entry num) total key len = 1.544835 GB avg = 0.028 KB total ukey len = 1.072173 GB avg = 0.015 KB total ukey num = 0.069265209 Billion total entry num = 0.056072938 Billion write speed all = 0.493005090 MB/sec (with seq num) write speed all = 0.472626587 MB/sec (without seq num) 2024/11/23-15:02:04.843033 139957181593152 [compaction_job.cc:1948] [default] [JOB 28676] Generated table #265183: 930877 keys, 37619618 bytes, temperature: kWarm 2024/11/23-15:02:04.843100 139957181593152 EVENT_LOG_v1 {"time_micros": 1732345324843066, "cf_name": "default", "job": 28676, "event": "table_file_creation", "file_number": 265183, "file_size": 37619618, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 4012776149, "table_properties": {"data_size": 31262080, "index_size": 1782400, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26064556, "raw_average_key_size": 28, "raw_value_size": 108153222, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930877, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345321, "oldest_key_time": 0, "file_creation_time": 1732345321, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265183, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:02:04.883717 139957181593152 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265184.sst) /dev/shm/ToplingTemp/Topling-85660-2csxep 2024/11/23-15:02:05.810489 139957181593152 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265184.sst): first pass time = 0.93's, 144.829'MB/sec 2024/11/23-15:02:07.113722 139957181593152 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265184.sst): old prealloc_size = 147639500, real_size = 32392224 2024/11/23-15:02:07.114296 139957181593152 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265184.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931619 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0304 GB avg-val = 116.07 avg-zval = 32.62 2024/11/23-15:02:07.115077 139957181593152 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265184.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:02:07.958793 139957181593152 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265184.sst): second pass time = 1.30's, 83.022'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.197'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3277.896'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.12's, 293.661'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31993.411'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.022'MB/sec zip pipeline throughput = 83.022'MB/sec entries = 931619 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.07 avg-zval = 32.62 usrkeys = 931619 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.07 avg-zval = 32.62 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931619 bytes = 3260688 } seq expand size = 6521333 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5136 value = 3.5585 dict = 1.00 all = 3.6768 } Zip/UnZip{ index = 0.0951 value = 0.2810 dict = 1.00 all = 0.2720 } ---------------------------- total value len = 9.422134 GB avg = 0.165 KB (by entry num) total key len = 1.570920 GB avg = 0.028 KB total ukey len = 1.090806 GB avg = 0.016 KB total ukey num = 0.070196828 Billion total entry num = 0.057004557 Billion write speed all = 0.499326958 MB/sec (with seq num) write speed all = 0.478612852 MB/sec (without seq num) 2024/11/23-15:02:08.002775 139957181593152 [compaction_job.cc:1948] [default] [JOB 28676] Generated table #265184: 931619 keys, 36503826 bytes, temperature: kWarm 2024/11/23-15:02:08.002845 139957181593152 EVENT_LOG_v1 {"time_micros": 1732345328002809, "cf_name": "default", "job": 28676, "event": "table_file_creation", "file_number": 265184, "file_size": 36503826, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3759280178, "largest_seqno": 4012776165, "table_properties": {"data_size": 30386896, "index_size": 1772224, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26085332, "raw_average_key_size": 28, "raw_value_size": 108132434, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931619, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345324, "oldest_key_time": 0, "file_creation_time": 1732345324, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265184, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:02:08.037347 139957181593152 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265185.sst) /dev/shm/ToplingTemp/Topling-85660-WbV2Lq 2024/11/23-15:02:09.031348 139957181593152 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265185.sst): first pass time = 0.99's, 135.033'MB/sec 2024/11/23-15:02:10.332909 139957181593152 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265185.sst): old prealloc_size = 147639500, real_size = 32978544 2024/11/23-15:02:10.333479 139957181593152 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265185.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930633 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.22 avg-zval = 33.28 2024/11/23-15:02:10.334283 139957181593152 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265185.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:02:11.190042 139957181593152 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265185.sst): second pass time = 1.30's, 83.154'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.272'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3238.624'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.12's, 297.222'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 37637.265'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.154'MB/sec zip pipeline throughput = 83.154'MB/sec entries = 930633 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.22 avg-zval = 33.28 usrkeys = 930633 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.22 avg-zval = 33.28 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930633 bytes = 3257232 } seq expand size = 6514431 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5150 value = 3.4918 dict = 1.00 all = 3.6190 } Zip/UnZip{ index = 0.0951 value = 0.2864 dict = 1.00 all = 0.2763 } ---------------------------- total value len = 9.536809 GB avg = 0.165 KB (by entry num) total key len = 1.596978 GB avg = 0.028 KB total ukey len = 1.109419 GB avg = 0.016 KB total ukey num = 0.071127461 Billion total entry num = 0.057935190 Billion write speed all = 0.505645090 MB/sec (with seq num) write speed all = 0.484595903 MB/sec (without seq num) 2024/11/23-15:02:11.235246 139957181593152 [compaction_job.cc:1948] [default] [JOB 28676] Generated table #265185: 930633 keys, 37087246 bytes, temperature: kWarm 2024/11/23-15:02:11.235319 139957181593152 EVENT_LOG_v1 {"time_micros": 1732345331235282, "cf_name": "default", "job": 28676, "event": "table_file_creation", "file_number": 265185, "file_size": 37087246, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3801926538, "largest_seqno": 4012776176, "table_properties": {"data_size": 30975584, "index_size": 1770112, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26057724, "raw_average_key_size": 28, "raw_value_size": 108160066, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930633, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345328, "oldest_key_time": 0, "file_creation_time": 1732345328, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265185, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:02:11.276552 139957181593152 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265186.sst) /dev/shm/ToplingTemp/Topling-85660-Dk1zUq 2024/11/23-15:02:12.268508 139957181593152 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265186.sst): first pass time = 0.99's, 135.311'MB/sec 2024/11/23-15:02:13.590160 139957181593152 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265186.sst): old prealloc_size = 147639500, real_size = 33776640 2024/11/23-15:02:13.590683 139957181593152 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265186.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 935117 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0318 GB avg-val = 115.53 avg-zval = 33.97 2024/11/23-15:02:13.591196 139957181593152 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265186.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:02:14.373797 139957181593152 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265186.sst): second pass time = 1.32's, 81.790'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.78's, 46.536'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3565.232'MB/sec (index lex order gen) rebuild zvType time = 0.66's, 0.355'MB/sec write SST data time = 0.12's, 291.810'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 25477.585'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.790'MB/sec zip pipeline throughput = 81.790'MB/sec entries = 935117 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.53 avg-zval = 33.97 usrkeys = 935117 avg-key = 20.00 avg-zkey = 1.90 avg-val = 119.28 avg-zval = 33.97 TagRS{ kinds = RS_Key0_TagN bytes = 146200 } TagArray{ size = 500501 bytes = 1689200 } seq expand size = 3503507 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5132 value = 3.4012 dict = 1.00 all = 3.6806 } Zip/UnZip{ index = 0.0951 value = 0.2940 dict = 1.00 all = 0.2717 } ---------------------------- total value len = 9.648347 GB avg = 0.164 KB (by entry num) total key len = 1.623161 GB avg = 0.028 KB total ukey len = 1.128121 GB avg = 0.016 KB total ukey num = 0.072062578 Billion total entry num = 0.058870307 Billion write speed all = 0.511825746 MB/sec (with seq num) write speed all = 0.490439902 MB/sec (without seq num) 2024/11/23-15:02:14.417846 139957181593152 [compaction_job.cc:1948] [default] [JOB 28676] Generated table #265186: 935117 keys, 36466067 bytes, temperature: kWarm 2024/11/23-15:02:14.417917 139957181593152 EVENT_LOG_v1 {"time_micros": 1732345334417881, "cf_name": "default", "job": 28676, "event": "table_file_creation", "file_number": 265186, "file_size": 36466067, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776192, "table_properties": {"data_size": 31763760, "index_size": 1778944, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26183276, "raw_average_key_size": 28, "raw_value_size": 108034482, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 935117, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345331, "oldest_key_time": 0, "file_creation_time": 1732345331, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265186, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:02:14.462138 139957181593152 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265187.sst) /dev/shm/ToplingTemp/Topling-85660-bvbpqp 2024/11/23-15:02:15.481585 139957181593152 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265187.sst): first pass time = 1.02's, 131.662'MB/sec 2024/11/23-15:02:16.881820 139957181593152 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265187.sst): old prealloc_size = 147639500, real_size = 35482240 2024/11/23-15:02:16.882458 139957181593152 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265187.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1080508 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1040 GB zip-val = 0.0331 GB avg-val = 96.22 avg-zval = 30.67 2024/11/23-15:02:16.882714 139957181593152 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265187.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:02:17.567062 139957181593152 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265187.sst): second pass time = 1.40's, 74.289'MB/sec, value only(77.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.69's, 53.776'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3365.374'MB/sec (index lex order gen) rebuild zvType time = 0.55's, 0.493'MB/sec write SST data time = 0.14's, 270.335'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.029'MB, throughput = 33228.576'MB/sec zip my value time = 1.40's, unzip length = 0.104'GB zip my value throughput = 74.289'MB/sec zip pipeline throughput = 74.289'MB/sec entries = 1080508 avg-key = 28.00 avg-zkey = 1.91 avg-val = 96.22 avg-zval = 30.67 usrkeys = 1080508 avg-key = 20.00 avg-zkey = 1.91 avg-val = 97.04 avg-zval = 30.67 TagRS{ kinds = RS_Key0_TagN bytes = 168920 } TagArray{ size = 126721 bytes = 427696 } seq expand size = 887047 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0216 GB value = 0.1040 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0331 GB dict = 1.03 MB all = 0.0368 GB } UnZip/Zip{ index = 10.4467 value = 3.1368 dict = 1.00 all = 3.6424 } Zip/UnZip{ index = 0.0957 value = 0.3188 dict = 1.00 all = 0.2745 } ---------------------------- total value len = 9.753198 GB avg = 0.163 KB (by entry num) total key len = 1.653415 GB avg = 0.028 KB total ukey len = 1.149731 GB avg = 0.016 KB total ukey num = 0.073143086 Billion total entry num = 0.059950815 Billion write speed all = 0.517885602 MB/sec (with seq num) write speed all = 0.496110399 MB/sec (without seq num) 2024/11/23-15:02:17.614497 139957181593152 [compaction_job.cc:1948] [default] [JOB 28676] Generated table #265187: 1080508 keys, 36848304 bytes, temperature: kWarm 2024/11/23-15:02:17.614565 139957181593152 EVENT_LOG_v1 {"time_micros": 1732345337614531, "cf_name": "default", "job": 28676, "event": "table_file_creation", "file_number": 265187, "file_size": 36848304, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3405782908, "table_properties": {"data_size": 33143344, "index_size": 2068608, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30254224, "raw_average_key_size": 28, "raw_value_size": 103963589, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1080508, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345334, "oldest_key_time": 0, "file_creation_time": 1732345334, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265187, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:02:17.652396 139957181593152 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265188.sst) /dev/shm/ToplingTemp/Topling-85660-bggEDp 2024/11/23-15:02:18.633896 139957181593152 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265188.sst): first pass time = 0.98's, 136.753'MB/sec 2024/11/23-15:02:19.911576 139957181593152 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265188.sst): old prealloc_size = 147639500, real_size = 30205376 2024/11/23-15:02:19.912145 139957181593152 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265188.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930493 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0282 GB avg-val = 116.24 avg-zval = 30.30 2024/11/23-15:02:19.913680 139957181593152 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265188.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:02:20.728678 139957181593152 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265188.sst): second pass time = 1.28's, 84.709'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.279'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3272.156'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.337'MB/sec write SST data time = 0.13's, 273.550'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34779.774'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.709'MB/sec zip pipeline throughput = 84.709'MB/sec entries = 930493 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.24 avg-zval = 30.30 usrkeys = 930493 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.24 avg-zval = 30.30 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930493 bytes = 3489360 } seq expand size = 6513451 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0282 GB dict = 1.08 MB all = 0.0345 GB } UnZip/Zip{ index = 10.4465 value = 3.8368 dict = 1.00 all = 3.8853 } Zip/UnZip{ index = 0.0957 value = 0.2606 dict = 1.00 all = 0.2574 } ---------------------------- total value len = 9.867875 GB avg = 0.162 KB (by entry num) total key len = 1.679469 GB avg = 0.028 KB total ukey len = 1.168341 GB avg = 0.016 KB total ukey num = 0.074073579 Billion total entry num = 0.060881308 Billion write speed all = 0.524199865 MB/sec (with seq num) write speed all = 0.502089864 MB/sec (without seq num) 2024/11/23-15:02:20.770554 139957181593152 [compaction_job.cc:1948] [default] [JOB 28676] Generated table #265188: 930493 keys, 34544614 bytes, temperature: kWarm 2024/11/23-15:02:20.770617 139957181593152 EVENT_LOG_v1 {"time_micros": 1732345340770583, "cf_name": "default", "job": 28676, "event": "table_file_creation", "file_number": 265188, "file_size": 34544614, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405782909, "largest_seqno": 4012775885, "table_properties": {"data_size": 28191152, "index_size": 1781440, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26053804, "raw_average_key_size": 28, "raw_value_size": 108163926, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930493, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345337, "oldest_key_time": 0, "file_creation_time": 1732345337, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265188, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:02:20.810396 139957181593152 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265189.sst) /dev/shm/ToplingTemp/Topling-85660-k98Kuo 2024/11/23-15:02:21.806380 139957181593152 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265189.sst): first pass time = 1.00's, 134.764'MB/sec 2024/11/23-15:02:23.095430 139957181593152 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265189.sst): old prealloc_size = 147639500, real_size = 30675120 2024/11/23-15:02:23.096041 139957181593152 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265189.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930646 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0287 GB avg-val = 116.22 avg-zval = 30.80 2024/11/23-15:02:23.097491 139957181593152 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265189.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:02:23.920766 139957181593152 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265189.sst): second pass time = 1.29's, 83.956'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.287'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3033.463'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.333'MB/sec write SST data time = 0.13's, 276.560'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33661.620'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.956'MB/sec zip pipeline throughput = 83.956'MB/sec entries = 930646 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.22 avg-zval = 30.80 usrkeys = 930646 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.22 avg-zval = 30.80 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930646 bytes = 3373600 } seq expand size = 6514522 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0287 GB dict = 1.08 MB all = 0.0349 GB } UnZip/Zip{ index = 10.4437 value = 3.7739 dict = 1.00 all = 3.8458 } Zip/UnZip{ index = 0.0958 value = 0.2650 dict = 1.00 all = 0.2600 } ---------------------------- total value len = 9.982549 GB avg = 0.161 KB (by entry num) total key len = 1.705527 GB avg = 0.028 KB total ukey len = 1.186954 GB avg = 0.016 KB total ukey num = 0.075004225 Billion total entry num = 0.061811954 Billion write speed all = 0.530511631 MB/sec (with seq num) write speed all = 0.508066904 MB/sec (without seq num) 2024/11/23-15:02:23.965008 139957181593152 [compaction_job.cc:1948] [default] [JOB 28676] Generated table #265189: 930646 keys, 34900254 bytes, temperature: kWarm 2024/11/23-15:02:23.965078 139957181593152 EVENT_LOG_v1 {"time_micros": 1732345343965043, "cf_name": "default", "job": 28676, "event": "table_file_creation", "file_number": 265189, "file_size": 34900254, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3487497056, "largest_seqno": 4012775902, "table_properties": {"data_size": 28660064, "index_size": 1782208, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26058088, "raw_average_key_size": 28, "raw_value_size": 108159664, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930646, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345340, "oldest_key_time": 0, "file_creation_time": 1732345340, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265189, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:02:24.006334 139957181593152 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265190.sst) /dev/shm/ToplingTemp/Topling-85660-L0elAq 2024/11/23-15:02:24.877068 139957181593152 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265190.sst): first pass time = 0.87's, 154.150'MB/sec 2024/11/23-15:02:26.152019 139957181593152 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265190.sst): old prealloc_size = 147639500, real_size = 30778608 2024/11/23-15:02:26.152555 139957181593152 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265190.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932240 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0288 GB avg-val = 115.97 avg-zval = 30.86 2024/11/23-15:02:26.153947 139957181593152 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265190.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:02:26.980114 139957181593152 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265190.sst): second pass time = 1.27's, 84.876'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.136'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3463.489'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.333'MB/sec write SST data time = 0.13's, 275.250'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 26712.806'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 84.876'MB/sec zip pipeline throughput = 84.876'MB/sec entries = 932240 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.97 avg-zval = 30.86 usrkeys = 932240 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.97 avg-zval = 30.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932240 bytes = 3262848 } seq expand size = 6525680 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0288 GB dict = 1.08 MB all = 0.0349 GB } UnZip/Zip{ index = 10.5130 value = 3.7577 dict = 1.00 all = 3.8466 } Zip/UnZip{ index = 0.0951 value = 0.2661 dict = 1.00 all = 0.2600 } ---------------------------- total value len = 10.097190 GB avg = 0.161 KB (by entry num) total key len = 1.731630 GB avg = 0.028 KB total ukey len = 1.205599 GB avg = 0.016 KB total ukey num = 0.075936465 Billion total entry num = 0.062744194 Billion write speed all = 0.536825310 MB/sec (with seq num) write speed all = 0.514045238 MB/sec (without seq num) 2024/11/23-15:02:27.022790 139957181593152 [compaction_job.cc:1948] [default] [JOB 28676] Generated table #265190: 932240 keys, 34892310 bytes, temperature: kWarm 2024/11/23-15:02:27.022861 139957181593152 EVENT_LOG_v1 {"time_micros": 1732345347022824, "cf_name": "default", "job": 28676, "event": "table_file_creation", "file_number": 265190, "file_size": 34892310, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3807172320, "largest_seqno": 4012775918, "table_properties": {"data_size": 28771872, "index_size": 1773504, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26102720, "raw_average_key_size": 28, "raw_value_size": 108115060, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932240, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345343, "oldest_key_time": 0, "file_creation_time": 1732345343, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265190, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:02:27.059667 139957181593152 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265191.sst) /dev/shm/ToplingTemp/Topling-85660-3PIzin 2024/11/23-15:02:27.884522 139957181593152 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265191.sst): first pass time = 0.82's, 162.724'MB/sec 2024/11/23-15:02:29.158195 139957181593152 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265191.sst): old prealloc_size = 147639500, real_size = 31668096 2024/11/23-15:02:29.158723 139957181593152 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265191.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925209 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0297 GB avg-val = 117.07 avg-zval = 32.06 2024/11/23-15:02:29.160253 139957181593152 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265191.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:02:29.974172 139957181593152 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265191.sst): second pass time = 1.27's, 85.130'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 44.111'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3504.801'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.12's, 288.376'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 37278.102'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 85.130'MB/sec zip pipeline throughput = 85.130'MB/sec entries = 925209 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.07 avg-zval = 32.06 usrkeys = 925209 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.07 avg-zval = 32.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925209 bytes = 3469552 } seq expand size = 6476463 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.4457 value = 3.6512 dict = 1.00 all = 3.7290 } Zip/UnZip{ index = 0.0957 value = 0.2739 dict = 1.00 all = 0.2682 } ---------------------------- total value len = 10.211978 GB avg = 0.160 KB (by entry num) total key len = 1.757536 GB avg = 0.028 KB total ukey len = 1.224103 GB avg = 0.016 KB total ukey num = 0.076861674 Billion total entry num = 0.063669403 Billion write speed all = 0.543136610 MB/sec (with seq num) write speed all = 0.520023770 MB/sec (without seq num) 2024/11/23-15:02:30.019886 139957181593152 [compaction_job.cc:1948] [default] [JOB 28676] Generated table #265191: 925209 keys, 35992948 bytes, temperature: kWarm 2024/11/23-15:02:30.019952 139957181593152 EVENT_LOG_v1 {"time_micros": 1732345350019917, "cf_name": "default", "job": 28676, "event": "table_file_creation", "file_number": 265191, "file_size": 35992948, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 4012775935, "table_properties": {"data_size": 29665136, "index_size": 1771456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25905852, "raw_average_key_size": 28, "raw_value_size": 108311882, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925209, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345347, "oldest_key_time": 0, "file_creation_time": 1732345347, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265191, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:02:30.056926 139957181593152 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265192.sst) /dev/shm/ToplingTemp/Topling-85660-6Kecvn 2024/11/23-15:02:31.113135 139957181593152 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265192.sst): first pass time = 1.06's, 127.079'MB/sec 2024/11/23-15:02:32.455616 139957181593152 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265192.sst): old prealloc_size = 147639500, real_size = 36026496 2024/11/23-15:02:32.456166 139957181593152 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265192.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925167 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0340 GB avg-val = 117.07 avg-zval = 36.78 2024/11/23-15:02:32.458252 139957181593152 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265192.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:02:33.285605 139957181593152 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265192.sst): second pass time = 1.34's, 80.760'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 48.479'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3438.819'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.329'MB/sec write SST data time = 0.13's, 318.237'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34387.858'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.760'MB/sec zip pipeline throughput = 80.760'MB/sec entries = 925167 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.07 avg-zval = 36.78 usrkeys = 925167 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.07 avg-zval = 36.78 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925167 bytes = 3353744 } seq expand size = 6476169 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0340 GB dict = 1.08 MB all = 0.0402 GB } UnZip/Zip{ index = 10.4434 value = 3.1835 dict = 1.00 all = 3.3358 } Zip/UnZip{ index = 0.0958 value = 0.3141 dict = 1.00 all = 0.2998 } ---------------------------- total value len = 10.326767 GB avg = 0.160 KB (by entry num) total key len = 1.783440 GB avg = 0.028 KB total ukey len = 1.242606 GB avg = 0.016 KB total ukey num = 0.077786841 Billion total entry num = 0.064594570 Billion write speed all = 0.549438272 MB/sec (with seq num) write speed all = 0.525993106 MB/sec (without seq num) 2024/11/23-15:02:33.334316 139957181593152 [compaction_job.cc:1948] [default] [JOB 28676] Generated table #265192: 925167 keys, 40235544 bytes, temperature: kWarm 2024/11/23-15:02:33.334387 139957181593152 EVENT_LOG_v1 {"time_micros": 1732345353334351, "cf_name": "default", "job": 28676, "event": "table_file_creation", "file_number": 265192, "file_size": 40235544, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775967, "table_properties": {"data_size": 34023216, "index_size": 1771776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25904676, "raw_average_key_size": 28, "raw_value_size": 108313086, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925167, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345350, "oldest_key_time": 0, "file_creation_time": 1732345350, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265192, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:02:33.373284 139957181593152 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265193.sst) /dev/shm/ToplingTemp/Topling-85660-pRP1bq 2024/11/23-15:02:34.419252 139957181593152 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265193.sst): first pass time = 1.05's, 128.324'MB/sec 2024/11/23-15:02:35.709494 139957181593152 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265193.sst): old prealloc_size = 147639500, real_size = 32821568 2024/11/23-15:02:35.710062 139957181593152 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265193.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928261 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0308 GB avg-val = 116.59 avg-zval = 33.21 2024/11/23-15:02:35.712204 139957181593152 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265193.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:02:36.555493 139957181593152 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265193.sst): second pass time = 1.29's, 83.933'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.921'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3222.455'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 295.698'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35719.191'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.933'MB/sec zip pipeline throughput = 83.933'MB/sec entries = 928261 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.59 avg-zval = 33.21 usrkeys = 928261 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.59 avg-zval = 33.21 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928261 bytes = 3480992 } seq expand size = 6497827 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5144 value = 3.5112 dict = 1.00 all = 3.6123 } Zip/UnZip{ index = 0.0951 value = 0.2848 dict = 1.00 all = 0.2768 } ---------------------------- total value len = 10.441492 GB avg = 0.159 KB (by entry num) total key len = 1.809432 GB avg = 0.028 KB total ukey len = 1.261171 GB avg = 0.016 KB total ukey num = 0.078715102 Billion total entry num = 0.065522831 Billion write speed all = 0.555740072 MB/sec (with seq num) write speed all = 0.531961513 MB/sec (without seq num) 2024/11/23-15:02:36.601172 139957181593152 [compaction_job.cc:1948] [default] [JOB 28676] Generated table #265193: 928261 keys, 37155988 bytes, temperature: kWarm 2024/11/23-15:02:36.601240 139957181593152 EVENT_LOG_v1 {"time_micros": 1732345356601205, "cf_name": "default", "job": 28676, "event": "table_file_creation", "file_number": 265193, "file_size": 37155988, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 4012776021, "table_properties": {"data_size": 30823600, "index_size": 1765696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25991308, "raw_average_key_size": 28, "raw_value_size": 108226494, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928261, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345353, "oldest_key_time": 0, "file_creation_time": 1732345353, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265193, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:02:36.636972 139957181593152 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265194.sst) /dev/shm/ToplingTemp/Topling-85660-R8Ho8m 2024/11/23-15:02:37.656126 139957181593152 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265194.sst): first pass time = 1.02's, 131.704'MB/sec 2024/11/23-15:02:38.947576 139957181593152 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265194.sst): old prealloc_size = 147639500, real_size = 32966432 2024/11/23-15:02:38.948206 139957181593152 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265194.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930517 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.24 avg-zval = 33.25 2024/11/23-15:02:38.948932 139957181593152 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265194.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:02:39.788577 139957181593152 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265194.sst): second pass time = 1.29's, 83.831'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.724'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2950.781'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 287.761'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34873.448'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.831'MB/sec zip pipeline throughput = 83.831'MB/sec entries = 930517 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.24 avg-zval = 33.25 usrkeys = 930517 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.86 avg-zval = 33.25 TagRS{ kinds = RS_Key0_TagN bytes = 145480 } TagArray{ size = 747520 bytes = 2803216 } seq expand size = 5232640 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.3782 value = 3.4959 dict = 1.00 all = 3.6501 } Zip/UnZip{ index = 0.0964 value = 0.2861 dict = 1.00 all = 0.2740 } ---------------------------- total value len = 10.554888 GB avg = 0.159 KB (by entry num) total key len = 1.835486 GB avg = 0.028 KB total ukey len = 1.279782 GB avg = 0.016 KB total ukey num = 0.079645619 Billion total entry num = 0.066453348 Billion write speed all = 0.561983555 MB/sec (with seq num) write speed all = 0.537870844 MB/sec (without seq num) 2024/11/23-15:02:39.848713 139957181593152 [compaction_job.cc:1948] [default] [JOB 28676] Generated table #265194: 930517 keys, 36770678 bytes, temperature: kWarm 2024/11/23-15:02:39.848800 139957181593152 EVENT_LOG_v1 {"time_micros": 1732345359848755, "cf_name": "default", "job": 28676, "event": "table_file_creation", "file_number": 265194, "file_size": 36770678, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776210, "table_properties": {"data_size": 30940432, "index_size": 1793216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26054476, "raw_average_key_size": 28, "raw_value_size": 108163302, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930517, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345356, "oldest_key_time": 0, "file_creation_time": 1732345356, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265194, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:02:39.896707 139957181593152 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265195.sst) /dev/shm/ToplingTemp/Topling-85660-moSh7o 2024/11/23-15:02:40.896221 139957181593152 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265195.sst): first pass time = 1.00's, 134.288'MB/sec 2024/11/23-15:02:42.180756 139957181593152 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265195.sst): old prealloc_size = 147639500, real_size = 30901824 2024/11/23-15:02:42.181303 139957181593152 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265195.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932563 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0289 GB avg-val = 115.92 avg-zval = 30.98 2024/11/23-15:02:42.182821 139957181593152 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265195.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:02:43.010709 139957181593152 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265195.sst): second pass time = 1.28's, 84.209'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.332'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3372.314'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.332'MB/sec write SST data time = 0.13's, 277.138'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 37429.990'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.209'MB/sec zip pipeline throughput = 84.209'MB/sec entries = 932563 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.92 avg-zval = 30.98 usrkeys = 932563 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.92 avg-zval = 30.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932563 bytes = 3380560 } seq expand size = 6527941 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.5128 value = 3.7414 dict = 1.00 all = 3.8203 } Zip/UnZip{ index = 0.0951 value = 0.2673 dict = 1.00 all = 0.2618 } ---------------------------- total value len = 10.669522 GB avg = 0.158 KB (by entry num) total key len = 1.861598 GB avg = 0.028 KB total ukey len = 1.298433 GB avg = 0.016 KB total ukey num = 0.080578182 Billion total entry num = 0.067385911 Billion write speed all = 0.568284230 MB/sec (with seq num) write speed all = 0.543836710 MB/sec (without seq num) 2024/11/23-15:02:43.053908 139957181593152 [compaction_job.cc:1948] [default] [JOB 28676] Generated table #265195: 932563 keys, 35132954 bytes, temperature: kWarm 2024/11/23-15:02:43.053987 139957181593152 EVENT_LOG_v1 {"time_micros": 1732345363053942, "cf_name": "default", "job": 28676, "event": "table_file_creation", "file_number": 265195, "file_size": 35132954, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776228, "table_properties": {"data_size": 28894384, "index_size": 1774144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26111764, "raw_average_key_size": 28, "raw_value_size": 108105978, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932563, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345359, "oldest_key_time": 0, "file_creation_time": 1732345359, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265195, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:02:43.089209 139957181593152 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265196.sst) /dev/shm/ToplingTemp/Topling-85660-2Weatq 2024/11/23-15:02:44.102272 139957181593152 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265196.sst): first pass time = 1.01's, 132.493'MB/sec 2024/11/23-15:02:45.406328 139957181593152 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265196.sst): old prealloc_size = 147639500, real_size = 32663120 2024/11/23-15:02:45.406904 139957181593152 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265196.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931926 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0306 GB avg-val = 116.02 avg-zval = 32.88 2024/11/23-15:02:45.408395 139957181593152 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265196.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:02:46.257945 139957181593152 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265196.sst): second pass time = 1.30's, 82.957'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.322'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3234.959'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.321'MB/sec write SST data time = 0.13's, 293.878'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 37110.599'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 82.957'MB/sec zip pipeline throughput = 82.957'MB/sec entries = 931926 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.02 avg-zval = 32.88 usrkeys = 931926 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.02 avg-zval = 32.88 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931926 bytes = 3378240 } seq expand size = 6523482 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4454 value = 3.5282 dict = 1.00 all = 3.6381 } Zip/UnZip{ index = 0.0957 value = 0.2834 dict = 1.00 all = 0.2749 } ---------------------------- total value len = 10.784169 GB avg = 0.158 KB (by entry num) total key len = 1.887692 GB avg = 0.028 KB total ukey len = 1.317071 GB avg = 0.016 KB total ukey num = 0.081510108 Billion total entry num = 0.068317837 Billion write speed all = 0.574582212 MB/sec (with seq num) write speed all = 0.549800239 MB/sec (without seq num) 2024/11/23-15:02:46.302568 139957181593152 [compaction_job.cc:1948] [default] [JOB 28676] Generated table #265196: 931926 keys, 36892388 bytes, temperature: kWarm 2024/11/23-15:02:46.302644 139957181593152 EVENT_LOG_v1 {"time_micros": 1732345366302602, "cf_name": "default", "job": 28676, "event": "table_file_creation", "file_number": 265196, "file_size": 36892388, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3518547401, "largest_seqno": 4012776268, "table_properties": {"data_size": 30645568, "index_size": 1784384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26093928, "raw_average_key_size": 28, "raw_value_size": 108123896, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931926, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345363, "oldest_key_time": 0, "file_creation_time": 1732345363, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265196, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:02:46.339859 139957181593152 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265197.sst) /dev/shm/ToplingTemp/Topling-85660-MFChUn 2024/11/23-15:02:47.362664 139957181593152 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265197.sst): first pass time = 1.02's, 131.230'MB/sec 2024/11/23-15:02:48.664907 139957181593152 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265197.sst): old prealloc_size = 147639500, real_size = 33859904 2024/11/23-15:02:48.665497 139957181593152 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265197.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922525 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0319 GB avg-val = 117.49 avg-zval = 34.54 2024/11/23-15:02:48.667008 139957181593152 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265197.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:02:49.504630 139957181593152 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265197.sst): second pass time = 1.30's, 83.282'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 45.466'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3100.528'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.323'MB/sec write SST data time = 0.13's, 304.938'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33375.069'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.282'MB/sec zip pipeline throughput = 83.282'MB/sec entries = 922525 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.49 avg-zval = 34.54 usrkeys = 922525 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.49 avg-zval = 34.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922525 bytes = 3459488 } seq expand size = 6457675 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.4426 value = 3.4017 dict = 1.00 all = 3.5156 } Zip/UnZip{ index = 0.0958 value = 0.2940 dict = 1.00 all = 0.2844 } ---------------------------- total value len = 10.899014 GB avg = 0.157 KB (by entry num) total key len = 1.913523 GB avg = 0.028 KB total ukey len = 1.335522 GB avg = 0.016 KB total ukey num = 0.082432633 Billion total entry num = 0.069240362 Billion write speed all = 0.580875368 MB/sec (with seq num) write speed all = 0.555762450 MB/sec (without seq num) 2024/11/23-15:02:49.551760 139957181593152 [compaction_job.cc:1948] [default] [JOB 28676] Generated table #265197: 922525 keys, 38177896 bytes, temperature: kWarm 2024/11/23-15:02:49.551832 139957181593152 EVENT_LOG_v1 {"time_micros": 1732345369551796, "cf_name": "default", "job": 28676, "event": "table_file_creation", "file_number": 265197, "file_size": 38177896, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776300, "table_properties": {"data_size": 31862256, "index_size": 1766848, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25830700, "raw_average_key_size": 28, "raw_value_size": 108387078, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922525, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345366, "oldest_key_time": 0, "file_creation_time": 1732345366, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265197, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:02:49.596040 139957181593152 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265198.sst) /dev/shm/ToplingTemp/Topling-85660-IlNFxp 2024/11/23-15:02:50.646926 139957181593152 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265198.sst): first pass time = 1.05's, 127.723'MB/sec 2024/11/23-15:02:51.968007 139957181593152 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265198.sst): old prealloc_size = 147639500, real_size = 33589712 2024/11/23-15:02:51.968581 139957181593152 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265198.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932783 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0316 GB avg-val = 115.89 avg-zval = 33.86 2024/11/23-15:02:51.970066 139957181593152 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265198.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:02:52.816612 139957181593152 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265198.sst): second pass time = 1.32's, 81.875'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.569'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3239.866'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.12's, 302.757'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 28024.495'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.875'MB/sec zip pipeline throughput = 81.875'MB/sec entries = 932783 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.89 avg-zval = 33.86 usrkeys = 932783 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.89 avg-zval = 33.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932783 bytes = 3381360 } seq expand size = 6529481 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.5138 value = 3.4228 dict = 1.00 all = 3.5487 } Zip/UnZip{ index = 0.0951 value = 0.2922 dict = 1.00 all = 0.2818 } ---------------------------- total value len = 11.013643 GB avg = 0.157 KB (by entry num) total key len = 1.939641 GB avg = 0.028 KB total ukey len = 1.354178 GB avg = 0.016 KB total ukey num = 0.083365416 Billion total entry num = 0.070173145 Billion write speed all = 0.587168192 MB/sec (with seq num) write speed all = 0.561720783 MB/sec (without seq num) 2024/11/23-15:02:52.863360 139957181593152 [compaction_job.cc:1948] [default] [JOB 28676] Generated table #265198: 932783 keys, 37821434 bytes, temperature: kWarm 2024/11/23-15:02:52.863428 139957181593152 EVENT_LOG_v1 {"time_micros": 1732345372863393, "cf_name": "default", "job": 28676, "event": "table_file_creation", "file_number": 265198, "file_size": 37821434, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776316, "table_properties": {"data_size": 31581952, "index_size": 1774400, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26117924, "raw_average_key_size": 28, "raw_value_size": 108099934, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932783, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345369, "oldest_key_time": 0, "file_creation_time": 1732345369, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265198, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:02:52.906060 139957181593152 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265199.sst) /dev/shm/ToplingTemp/Topling-85660-YBDXPp 2024/11/23-15:02:53.888322 139957181593152 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265199.sst): first pass time = 0.98's, 136.649'MB/sec 2024/11/23-15:02:55.188324 139957181593152 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265199.sst): old prealloc_size = 147639500, real_size = 32786544 2024/11/23-15:02:55.188862 139957181593152 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265199.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926271 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0308 GB avg-val = 116.90 avg-zval = 33.24 2024/11/23-15:02:55.190420 139957181593152 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265199.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:02:56.029833 139957181593152 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265199.sst): second pass time = 1.30's, 83.345'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.106'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3419.789'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.12's, 300.174'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34928.329'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.345'MB/sec zip pipeline throughput = 83.345'MB/sec entries = 926271 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.90 avg-zval = 33.24 usrkeys = 926271 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.90 avg-zval = 33.24 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926271 bytes = 3473536 } seq expand size = 6483897 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5128 value = 3.5165 dict = 1.00 all = 3.6163 } Zip/UnZip{ index = 0.0951 value = 0.2844 dict = 1.00 all = 0.2765 } ---------------------------- total value len = 11.128409 GB avg = 0.157 KB (by entry num) total key len = 1.965576 GB avg = 0.028 KB total ukey len = 1.372703 GB avg = 0.016 KB total ukey num = 0.084291687 Billion total entry num = 0.071099416 Billion write speed all = 0.593459715 MB/sec (with seq num) write speed all = 0.567680160 MB/sec (without seq num) 2024/11/23-15:02:56.076068 139957181593152 [compaction_job.cc:1948] [default] [JOB 28676] Generated table #265199: 926271 keys, 37114822 bytes, temperature: kWarm 2024/11/23-15:02:56.076138 139957181593152 EVENT_LOG_v1 {"time_micros": 1732345376076102, "cf_name": "default", "job": 28676, "event": "table_file_creation", "file_number": 265199, "file_size": 37114822, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776344, "table_properties": {"data_size": 30792608, "index_size": 1762176, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25935588, "raw_average_key_size": 28, "raw_value_size": 108282214, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926271, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345372, "oldest_key_time": 0, "file_creation_time": 1732345372, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265199, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:02:56.113222 139957181593152 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265200.sst) /dev/shm/ToplingTemp/Topling-85660-NlSPjr 2024/11/23-15:02:57.124165 139957181593152 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265200.sst): first pass time = 1.01's, 132.771'MB/sec 2024/11/23-15:02:58.429018 139957181593152 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265200.sst): old prealloc_size = 147639500, real_size = 33060416 2024/11/23-15:02:58.429574 139957181593152 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265200.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924957 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 117.11 avg-zval = 33.56 2024/11/23-15:02:58.431132 139957181593152 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265200.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:02:59.273649 139957181593152 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265200.sst): second pass time = 1.30's, 83.063'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.263'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3344.237'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.12's, 301.247'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 26283.637'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.063'MB/sec zip pipeline throughput = 83.063'MB/sec entries = 924957 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.11 avg-zval = 33.56 usrkeys = 924957 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.11 avg-zval = 33.56 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924957 bytes = 3468608 } seq expand size = 6474699 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.3751 value = 3.4890 dict = 1.00 all = 3.5902 } Zip/UnZip{ index = 0.0964 value = 0.2866 dict = 1.00 all = 0.2785 } ---------------------------- total value len = 11.243203 GB avg = 0.156 KB (by entry num) total key len = 1.991475 GB avg = 0.028 KB total ukey len = 1.391202 GB avg = 0.016 KB total ukey num = 0.085216644 Billion total entry num = 0.072024373 Billion write speed all = 0.599748154 MB/sec (with seq num) write speed all = 0.573637063 MB/sec (without seq num) 2024/11/23-15:02:59.318893 139957181593152 [compaction_job.cc:1948] [default] [JOB 28676] Generated table #265200: 924957 keys, 37384438 bytes, temperature: kWarm 2024/11/23-15:02:59.318964 139957181593152 EVENT_LOG_v1 {"time_micros": 1732345379318929, "cf_name": "default", "job": 28676, "event": "table_file_creation", "file_number": 265200, "file_size": 37384438, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012771944, "table_properties": {"data_size": 31045936, "index_size": 1783040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25898796, "raw_average_key_size": 28, "raw_value_size": 108318934, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924957, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345376, "oldest_key_time": 0, "file_creation_time": 1732345376, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265200, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:02:59.354452 139957181593152 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265201.sst) /dev/shm/ToplingTemp/Topling-85660-KSyhdq 2024/11/23-15:03:00.364318 139957181593152 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265201.sst): first pass time = 1.01's, 132.912'MB/sec 2024/11/23-15:03:01.829458 139957181593152 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265201.sst): old prealloc_size = 147639500, real_size = 35913440 2024/11/23-15:03:01.830092 139957181593152 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265201.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930405 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0339 GB avg-val = 116.26 avg-zval = 36.45 2024/11/23-15:03:01.830422 139957181593152 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265201.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:03:02.568446 139957181593152 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265201.sst): second pass time = 1.46's, 73.867'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.74's, 51.156'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2898.146'MB/sec (index lex order gen) rebuild zvType time = 0.61's, 0.380'MB/sec write SST data time = 0.13's, 300.499'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35214.384'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 73.867'MB/sec zip pipeline throughput = 73.867'MB/sec entries = 930405 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.26 avg-zval = 36.45 usrkeys = 930405 avg-key = 20.00 avg-zkey = 1.90 avg-val = 118.31 avg-zval = 36.45 TagRS{ kinds = RS_Key0_TagN bytes = 145480 } TagArray{ size = 273007 bytes = 887280 } seq expand size = 1911049 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0339 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.5124 value = 3.1898 dict = 1.00 all = 3.5505 } Zip/UnZip{ index = 0.0951 value = 0.3135 dict = 1.00 all = 0.2816 } ---------------------------- total value len = 11.353281 GB avg = 0.156 KB (by entry num) total key len = 2.017526 GB avg = 0.028 KB total ukey len = 1.409810 GB avg = 0.016 KB total ukey num = 0.086147049 Billion total entry num = 0.072954778 Billion write speed all = 0.605826573 MB/sec (with seq num) write speed all = 0.579382129 MB/sec (without seq num) 2024/11/23-15:03:02.616326 139957181593152 [compaction_job.cc:1948] [default] [JOB 28676] Generated table #265201: 930405 keys, 37802305 bytes, temperature: kWarm 2024/11/23-15:03:02.616398 139957181593152 EVENT_LOG_v1 {"time_micros": 1732345382616362, "cf_name": "default", "job": 28676, "event": "table_file_creation", "file_number": 265201, "file_size": 37802305, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 33910544, "index_size": 1770112, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26051340, "raw_average_key_size": 28, "raw_value_size": 108166522, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930405, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345379, "oldest_key_time": 0, "file_creation_time": 1732345379, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265201, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:03:02.652575 139957181593152 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265202.sst) /dev/shm/ToplingTemp/Topling-85660-uDcuYq 2024/11/23-15:03:03.590952 139957181593152 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265202.sst): first pass time = 0.94's, 143.038'MB/sec 2024/11/23-15:03:05.086410 139957181593152 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265202.sst): old prealloc_size = 147639500, real_size = 37305104 2024/11/23-15:03:05.087015 139957181593152 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265202.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925707 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0353 GB avg-val = 116.99 avg-zval = 38.13 2024/11/23-15:03:05.087371 139957181593152 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265202.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:03:05.814400 139957181593152 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265202.sst): second pass time = 1.49's, 72.486'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.73's, 54.074'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3016.472'MB/sec (index lex order gen) rebuild zvType time = 0.60's, 0.384'MB/sec write SST data time = 0.12's, 315.766'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33364.319'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.486'MB/sec zip pipeline throughput = 72.486'MB/sec entries = 925707 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.99 avg-zval = 38.13 usrkeys = 925707 avg-key = 20.00 avg-zkey = 1.91 avg-val = 119.12 avg-zval = 38.13 TagRS{ kinds = RS_Key0_TagN bytes = 144760 } TagArray{ size = 281512 bytes = 1055680 } seq expand size = 1970584 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0353 GB dict = 1.08 MB all = 0.0394 GB } UnZip/Zip{ index = 10.4461 value = 3.0678 dict = 1.00 all = 3.4097 } Zip/UnZip{ index = 0.0957 value = 0.3260 dict = 1.00 all = 0.2933 } ---------------------------- total value len = 11.463549 GB avg = 0.155 KB (by entry num) total key len = 2.043446 GB avg = 0.028 KB total ukey len = 1.428324 GB avg = 0.016 KB total ukey num = 0.087072756 Billion total entry num = 0.073880485 Billion write speed all = 0.611907223 MB/sec (with seq num) write speed all = 0.585131170 MB/sec (without seq num) 2024/11/23-15:03:05.863134 139957181593152 [compaction_job.cc:1948] [default] [JOB 28676] Generated table #265202: 925707 keys, 39364033 bytes, temperature: kWarm 2024/11/23-15:03:05.863205 139957181593152 EVENT_LOG_v1 {"time_micros": 1732345385863169, "cf_name": "default", "job": 28676, "event": "table_file_creation", "file_number": 265202, "file_size": 39364033, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4037360331, "table_properties": {"data_size": 35301120, "index_size": 1772352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25919796, "raw_average_key_size": 28, "raw_value_size": 108297942, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925707, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345382, "oldest_key_time": 0, "file_creation_time": 1732345382, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265202, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:03:05.901128 139957181593152 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265203.sst) /dev/shm/ToplingTemp/Topling-85660-rfP6qn 2024/11/23-15:03:06.974737 139957181593152 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265203.sst): first pass time = 1.07's, 125.020'MB/sec 2024/11/23-15:03:08.295162 139957181593152 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265203.sst): old prealloc_size = 147639500, real_size = 37295296 2024/11/23-15:03:08.295684 139957181593152 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265203.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925378 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0353 GB avg-val = 117.04 avg-zval = 38.15 2024/11/23-15:03:08.296396 139957181593152 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265203.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:03:09.101433 139957181593152 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265203.sst): second pass time = 1.32's, 82.097'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 51.048'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3523.257'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.340'MB/sec write SST data time = 0.12's, 330.572'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34744.557'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.097'MB/sec zip pipeline throughput = 82.097'MB/sec entries = 925378 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.04 avg-zval = 38.15 usrkeys = 925378 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.04 avg-zval = 38.15 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925378 bytes = 3007488 } seq expand size = 6477646 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0353 GB dict = 1.08 MB all = 0.0412 GB } UnZip/Zip{ index = 10.5130 value = 3.0679 dict = 1.00 all = 3.2611 } Zip/UnZip{ index = 0.0951 value = 0.3260 dict = 1.00 all = 0.3066 } ---------------------------- total value len = 11.578334 GB avg = 0.155 KB (by entry num) total key len = 2.069357 GB avg = 0.028 KB total ukey len = 1.446832 GB avg = 0.016 KB total ukey num = 0.087998134 Billion total entry num = 0.074805863 Billion write speed all = 0.618189088 MB/sec (with seq num) write speed all = 0.591081692 MB/sec (without seq num) 2024/11/23-15:03:09.151463 139957181593152 [compaction_job.cc:1948] [default] [JOB 28676] Generated table #265203: 925378 keys, 41157710 bytes, temperature: kWarm 2024/11/23-15:03:09.151536 139957181593152 EVENT_LOG_v1 {"time_micros": 1732345389151500, "cf_name": "default", "job": 28676, "event": "table_file_creation", "file_number": 265203, "file_size": 41157710, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4012776751, "largest_seqno": 4048637100, "table_properties": {"data_size": 35303344, "index_size": 1760448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25910584, "raw_average_key_size": 28, "raw_value_size": 108307148, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925378, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345385, "oldest_key_time": 0, "file_creation_time": 1732345385, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265203, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:03:09.187505 139957181593152 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265204.sst) /dev/shm/ToplingTemp/Topling-85660-4HJnDq 2024/11/23-15:03:09.280423 139957181593152 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265204.sst): first pass time = 0.09's, 239.452'MB/sec 2024/11/23-15:03:09.500197 139957181593152 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265204.sst): old prealloc_size = 147639500, real_size = 6623296 2024/11/23-15:03:09.500334 139957181593152 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265204.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 153238 prefix = 4 raw-key = 0.0025 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0179 GB zip-val = 0.0063 GB avg-val = 117.14 avg-zval = 41.06 2024/11/23-15:03:09.500486 139957181593152 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265204.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.022 GB 2024/11/23-15:03:09.635384 139957181593152 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00/265204.sst): second pass time = 0.22's, 81.940'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.14's, 53.526'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2550.436'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.340'MB/sec write SST data time = 0.02's, 325.481'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.188'MB, throughput = 6161.457'MB/sec zip my value time = 0.22's, unzip length = 0.018'GB zip my value throughput = 81.940'MB/sec zip pipeline throughput = 81.940'MB/sec entries = 153238 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.14 avg-zval = 41.06 usrkeys = 153238 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.14 avg-zval = 41.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 153238 bytes = 459728 } seq expand size = 1072666 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0031 GB value = 0.0179 GB dict = 0.19 MB all = 0.0222 GB } __ZipSize{ index = 0.0003 GB value = 0.0063 GB dict = 0.19 MB all = 0.0072 GB } UnZip/Zip{ index = 10.4648 value = 2.8528 dict = 1.00 all = 3.0741 } Zip/UnZip{ index = 0.0956 value = 0.3505 dict = 1.00 all = 0.3253 } ---------------------------- total value len = 11.597356 GB avg = 0.155 KB (by entry num) total key len = 2.073647 GB avg = 0.028 KB total ukey len = 1.449897 GB avg = 0.016 KB total ukey num = 0.088151372 Billion total entry num = 0.074959101 Billion write speed all = 0.619230110 MB/sec (with seq num) write speed all = 0.592067843 MB/sec (without seq num) 2024/11/23-15:03:09.650044 139957181593152 [compaction_job.cc:1948] [default] [JOB 28676] Generated table #265204: 153238 keys, 7234795 bytes, temperature: kWarm 2024/11/23-15:03:09.650106 139957181593152 EVENT_LOG_v1 {"time_micros": 1732345389650073, "cf_name": "default", "job": 28676, "event": "table_file_creation", "file_number": 265204, "file_size": 7234795, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4031991610, "largest_seqno": 4048636431, "table_properties": {"data_size": 6291952, "index_size": 292864, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4290664, "raw_average_key_size": 28, "raw_value_size": 17949840, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 153238, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345389, "oldest_key_time": 0, "file_creation_time": 1732345389, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265204, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:03:09.661737 139957181593152 [compaction_job.cc:703] job-28676: subcompact[0], size: 0.904858 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28676/att-00 [265180,265181,265182,265183,265184,265185,265186,265187,265188,265189,265190,265191,265192,265193,265194,265195,265196,265197,265198,265199,265200,265201,265202,265203,265204] 2024/11/23-15:03:09.802458 139957181593152 (Original Log Time 2024/11/23-15:01:51.457433) EVENT_LOG_v1 {"time_micros": 1732345311457405, "job": 28676, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [265135], "files_L3": [265077, 265078, 265079, 265080, 265081, 265082, 265083, 265084, 265085, 265086, 265087, 265088, 265089, 265090, 265091, 265092, 265093, 265094, 265095, 265096, 265097, 265098, 265099, 265147], "score": 0.997776, "input_data_size": 927212199, "oldest_snapshot_seqno": 4049168726} 2024/11/23-15:03:09.802462 139957181593152 (Original Log Time 2024/11/23-15:03:09.800883) [compaction_job.cc:2022] [default] [JOB 28676] Compacted 1@2 + 24@3 files to L3 => 904858434 bytes 2024/11/23-15:03:09.802463 139957181593152 (Original Log Time 2024/11/23-15:03:09.802384) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 25 0 0 0] max score 0.03, MB/sec: 11.9 rd, 11.6 wr, level 3, files in(1, 24) out(25 +0 blob) MB in(27.1, 857.2 +0.0 blob) out(862.9 +0.0 blob), read-write-amplify(64.5) write-amplify(31.8) OK, records in: 22680287, records dropped: 79350 output_compression: Snappy 2024/11/23-15:03:09.802466 139957181593152 (Original Log Time 2024/11/23-15:03:09.802432) EVENT_LOG_v1 {"time_micros": 1732345389802409, "cf": "default", "job": 28676, "event": "compaction_finished", "compaction_time_micros": 78204310, "compaction_time_cpu_micros": 33280245, "output_level": 3, "num_output_files": 25, "total_output_size": 904858434, "num_input_records": 22680287, "num_output_records": 22600937, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 612778361, "file_range_sync_nanos": 0, "file_fsync_nanos": 17790, "file_prepare_write_nanos": 66326, "lsm_state": [0, 0, 0, 25, 0, 0, 0]} 2024/11/23-15:03:09.919632 139957181593152 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732345311,"executesMs":78478,"compactionJobId":28676,"attempt":0,"compactionInputRawBytes":3254734891,"compactionInputZipBytes":927212199,"compactionOutputRawBytes":0,"compactionOutputZipBytes":904858434,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-5836bba34dc4807cf9a59677a7951d99-e25b1720245c303d-00"}}, response = 2024/11/23-15:03:09.919737 139957181593152 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28676/att-00: olev 3, work 78.479 s, result 7.732 ms, install 1.599 ms, input{raw 3.031 GiB zip 884.258 MiB}