2024/11/23-19:17:25.209566 139957231949376 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00: fork to child time = 0.007811 sec 2024/11/23-19:17:25.210386 139957231949376 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00 2024/11/23-19:17:25.217830 139957231949376 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00 2024/11/23-19:17:25.218744 139957231949376 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-19:17:25.219030 139957231949376 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-19:17:25.272682 139957231949376 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-34027/att-00/MANIFEST-206347 2024/11/23-19:17:25.306243 139957231949376 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-34027/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 276822, last_sequence is 4210371674, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 276660 2024/11/23-19:17:25.306252 139957231949376 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-19:17:25.306394 139957231949376 [version_set.cc:5721] Creating manifest 276822 2024/11/23-19:17:25.339157 139957231949376 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-19:17:25.364679 139957231949376 [compaction_job.cc:2437] [default] [JOB 34027] Compacting 1@2 + 22@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-19:17:25.364693 139957231949376 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 25476 Base level 2, inputs: [276789(31MB)], [276716(40MB) 276717(40MB) 276718(40MB) 276719(40MB) 276720(40MB) 276721(40MB) 276722(40MB) 276723(40MB) 276724(40MB) 276725(40MB) 276726(40MB) 276727(40MB) 276728(40MB) 276729(40MB) 276730(40MB) 276731(40MB) 276732(40MB) 276733(40MB) 276734(40MB) 276735(40MB) 276736(40MB) 276819(40MB)] 2024/11/23-19:17:25.371739 139957231949376 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-19:17:25.371767 139957231949376 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-19:17:25.371786 139957231949376 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-19:17:25.371792 139957231949376 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-19:17:25.371802 139957231949376 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-19:17:26.139520 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276823.sst) /dev/shm/ToplingTemp/Topling-163366-GjS1hb 2024/11/23-19:17:26.311579 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276823.sst): first pass time = 0.17's, 780.257'MB/sec 2024/11/23-19:17:27.122804 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276823.sst): old prealloc_size = 147639500, real_size = 40597984 2024/11/23-19:17:27.122926 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276823.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.99 avg-zval = 311.62 2024/11/23-19:17:27.123311 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276823.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:17:27.286542 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276823.sst): second pass time = 0.81's, 162.584'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 259.162'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2330.956'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.05's, 925.689'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 36440.511'MB/sec zip my value time = 0.81's, unzip length = 0.132'GB zip my value throughput = 162.584'MB/sec zip pipeline throughput = 162.584'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1016.99 avg-zval = 311.62 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1023.99 avg-zval = 311.62 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6744 value = 3.2636 dict = 1.00 all = 3.1632 } Zip/UnZip{ index = 0.1498 value = 0.3064 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 8.981155 GB avg = 0.171 KB (by entry num) total key len = 1.443410 GB avg = 0.028 KB total ukey len = 0.999431 GB avg = 0.015 KB total ukey num = 0.065679865 Billion total entry num = 0.052487594 Billion write speed all = 0.279216408 MB/sec (with seq num) write speed all = 0.267969590 MB/sec (without seq num) 2024/11/23-19:17:27.342552 139957231949376 [compaction_job.cc:1948] [default] [JOB 34027] Generated table #276823: 129430 keys, 42430988 bytes, temperature: kWarm 2024/11/23-19:17:27.342669 139957231949376 EVENT_LOG_v1 {"time_micros": 1732360647342598, "cf_name": "default", "job": 34027, "event": "table_file_creation", "file_number": 276823, "file_size": 42430988, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237877277, "largest_seqno": 4210211432, "table_properties": {"data_size": 40332752, "index_size": 232704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629181, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360646, "oldest_key_time": 0, "file_creation_time": 1732360646, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276823, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:17:27.406061 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276824.sst) /dev/shm/ToplingTemp/Topling-163366-4OW71a 2024/11/23-19:17:28.146411 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276824.sst): first pass time = 0.74's, 181.303'MB/sec 2024/11/23-19:17:28.961428 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276824.sst): old prealloc_size = 147639500, real_size = 40868560 2024/11/23-19:17:28.961567 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276824.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129434 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.74 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1016.96 avg-zval = 313.76 2024/11/23-19:17:28.961727 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276824.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:17:29.125646 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276824.sst): second pass time = 0.81's, 161.654'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 260.084'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2054.938'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.273'MB/sec write SST data time = 0.05's, 934.969'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 43828.250'MB/sec zip my value time = 0.81's, unzip length = 0.132'GB zip my value throughput = 161.654'MB/sec zip pipeline throughput = 161.654'MB/sec entries = 129434 avg-key = 20.00 avg-zkey = 1.74 avg-val = 1016.96 avg-zval = 313.76 usrkeys = 129434 avg-key = 12.00 avg-zkey = 1.74 avg-val = 1023.96 avg-zval = 313.76 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129434 bytes = 485392 } seq expand size = 906038 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.9162 value = 3.2412 dict = 1.00 all = 3.1432 } Zip/UnZip{ index = 0.1446 value = 0.3085 dict = 1.00 all = 0.3181 } ---------------------------- total value len = 9.113690 GB avg = 0.173 KB (by entry num) total key len = 1.445998 GB avg = 0.027 KB total ukey len = 1.000984 GB avg = 0.015 KB total ukey num = 0.065809299 Billion total entry num = 0.052617028 Billion write speed all = 0.282821700 MB/sec (with seq num) write speed all = 0.271547703 MB/sec (without seq num) 2024/11/23-19:17:29.210849 139957231949376 [compaction_job.cc:1948] [default] [JOB 34027] Generated table #276824: 129434 keys, 42701331 bytes, temperature: kWarm 2024/11/23-19:17:29.210897 139957231949376 EVENT_LOG_v1 {"time_micros": 1732360649210869, "cf_name": "default", "job": 34027, "event": "table_file_creation", "file_number": 276824, "file_size": 42701331, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868195, "largest_seqno": 4210200951, "table_properties": {"data_size": 40611456, "index_size": 224576, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588680, "raw_average_key_size": 20, "raw_value_size": 131629248, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129434, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360647, "oldest_key_time": 0, "file_creation_time": 1732360647, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276824, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:17:29.252599 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276825.sst) /dev/shm/ToplingTemp/Topling-163366-FnGf2b 2024/11/23-19:17:29.784835 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276825.sst): first pass time = 0.53's, 252.192'MB/sec 2024/11/23-19:17:30.607944 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276825.sst): old prealloc_size = 147639500, real_size = 40581552 2024/11/23-19:17:30.608102 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276825.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129433 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.97 avg-zval = 311.48 2024/11/23-19:17:30.608275 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276825.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:17:30.771661 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276825.sst): second pass time = 0.82's, 160.253'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 259.105'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2050.906'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.274'MB/sec write SST data time = 0.05's, 932.358'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45292.418'MB/sec zip my value time = 0.82's, unzip length = 0.132'GB zip my value throughput = 160.253'MB/sec zip pipeline throughput = 160.253'MB/sec entries = 129433 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1016.97 avg-zval = 311.48 usrkeys = 129433 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1023.97 avg-zval = 311.48 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129433 bytes = 485392 } seq expand size = 906031 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6746 value = 3.2649 dict = 1.00 all = 3.1644 } Zip/UnZip{ index = 0.1498 value = 0.3063 dict = 1.00 all = 0.3160 } ---------------------------- total value len = 9.246226 GB avg = 0.175 KB (by entry num) total key len = 1.448587 GB avg = 0.027 KB total ukey len = 1.002538 GB avg = 0.015 KB total ukey num = 0.065938732 Billion total entry num = 0.052746461 Billion write speed all = 0.286428121 MB/sec (with seq num) write speed all = 0.275126889 MB/sec (without seq num) 2024/11/23-19:17:30.824582 139957231949376 [compaction_job.cc:1948] [default] [JOB 34027] Generated table #276825: 129433 keys, 42414554 bytes, temperature: kWarm 2024/11/23-19:17:30.824652 139957231949376 EVENT_LOG_v1 {"time_micros": 1732360650824616, "cf_name": "default", "job": 34027, "event": "table_file_creation", "file_number": 276825, "file_size": 42414554, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237873554, "largest_seqno": 4210208392, "table_properties": {"data_size": 40316320, "index_size": 232704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588660, "raw_average_key_size": 20, "raw_value_size": 131629407, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129433, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360649, "oldest_key_time": 0, "file_creation_time": 1732360649, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276825, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:17:30.867151 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276826.sst) /dev/shm/ToplingTemp/Topling-163366-4lC5Jc 2024/11/23-19:17:31.526594 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276826.sst): first pass time = 0.66's, 203.551'MB/sec 2024/11/23-19:17:32.335788 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276826.sst): old prealloc_size = 147639500, real_size = 40873376 2024/11/23-19:17:32.335905 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276826.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129435 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.74 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1016.96 avg-zval = 313.80 2024/11/23-19:17:32.336078 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276826.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:17:32.495618 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276826.sst): second pass time = 0.81's, 162.830'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 267.238'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2320.216'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.276'MB/sec write SST data time = 0.04's, 1010.115'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45014.538'MB/sec zip my value time = 0.81's, unzip length = 0.132'GB zip my value throughput = 162.830'MB/sec zip pipeline throughput = 162.830'MB/sec entries = 129435 avg-key = 20.00 avg-zkey = 1.74 avg-val = 1016.96 avg-zval = 313.80 usrkeys = 129435 avg-key = 12.00 avg-zkey = 1.74 avg-val = 1023.96 avg-zval = 313.80 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129435 bytes = 485392 } seq expand size = 906045 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.9162 value = 3.2408 dict = 1.00 all = 3.1429 } Zip/UnZip{ index = 0.1446 value = 0.3086 dict = 1.00 all = 0.3182 } ---------------------------- total value len = 9.378762 GB avg = 0.177 KB (by entry num) total key len = 1.451176 GB avg = 0.027 KB total ukey len = 1.004091 GB avg = 0.015 KB total ukey num = 0.066068167 Billion total entry num = 0.052875896 Billion write speed all = 0.290033636 MB/sec (with seq num) write speed all = 0.278705195 MB/sec (without seq num) 2024/11/23-19:17:32.547630 139957231949376 [compaction_job.cc:1948] [default] [JOB 34027] Generated table #276826: 129435 keys, 42705977 bytes, temperature: kWarm 2024/11/23-19:17:32.547692 139957231949376 EVENT_LOG_v1 {"time_micros": 1732360652547658, "cf_name": "default", "job": 34027, "event": "table_file_creation", "file_number": 276826, "file_size": 42705977, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868596, "largest_seqno": 4210152751, "table_properties": {"data_size": 40616272, "index_size": 224576, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588700, "raw_average_key_size": 20, "raw_value_size": 131629996, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129435, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360650, "oldest_key_time": 0, "file_creation_time": 1732360650, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276826, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:17:32.588046 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276827.sst) /dev/shm/ToplingTemp/Topling-163366-95Gtu9 2024/11/23-19:17:33.117852 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276827.sst): first pass time = 0.53's, 253.349'MB/sec 2024/11/23-19:17:33.943263 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276827.sst): old prealloc_size = 147639500, real_size = 40584304 2024/11/23-19:17:33.943396 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276827.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129432 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.98 avg-zval = 311.51 2024/11/23-19:17:33.943560 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276827.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:17:34.104957 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276827.sst): second pass time = 0.82's, 159.609'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 262.363'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2107.273'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.278'MB/sec write SST data time = 0.05's, 942.257'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 43533.851'MB/sec zip my value time = 0.82's, unzip length = 0.132'GB zip my value throughput = 159.609'MB/sec zip pipeline throughput = 159.609'MB/sec entries = 129432 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1016.98 avg-zval = 311.51 usrkeys = 129432 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1023.98 avg-zval = 311.51 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129432 bytes = 485392 } seq expand size = 906024 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6745 value = 3.2647 dict = 1.00 all = 3.1642 } Zip/UnZip{ index = 0.1498 value = 0.3063 dict = 1.00 all = 0.3160 } ---------------------------- total value len = 9.511297 GB avg = 0.179 KB (by entry num) total key len = 1.453764 GB avg = 0.027 KB total ukey len = 1.005644 GB avg = 0.015 KB total ukey num = 0.066197599 Billion total entry num = 0.053005328 Billion write speed all = 0.293639696 MB/sec (with seq num) write speed all = 0.282284014 MB/sec (without seq num) 2024/11/23-19:17:34.156029 139957231949376 [compaction_job.cc:1948] [default] [JOB 34027] Generated table #276827: 129432 keys, 42417080 bytes, temperature: kWarm 2024/11/23-19:17:34.156097 139957231949376 EVENT_LOG_v1 {"time_micros": 1732360654156062, "cf_name": "default", "job": 34027, "event": "table_file_creation", "file_number": 276827, "file_size": 42417080, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237869923, "largest_seqno": 4210210420, "table_properties": {"data_size": 40319072, "index_size": 232704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588640, "raw_average_key_size": 20, "raw_value_size": 131629176, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360652, "oldest_key_time": 0, "file_creation_time": 1732360652, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276827, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:17:34.193180 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276828.sst) /dev/shm/ToplingTemp/Topling-163366-Wq1nAb 2024/11/23-19:17:34.956401 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276828.sst): first pass time = 0.76's, 175.870'MB/sec 2024/11/23-19:17:35.765390 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276828.sst): old prealloc_size = 147639500, real_size = 40588864 2024/11/23-19:17:35.765515 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276828.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.00 avg-zval = 311.55 2024/11/23-19:17:35.765679 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276828.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:17:35.925833 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276828.sst): second pass time = 0.81's, 162.871'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 264.435'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2263.944'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.04's, 998.174'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 49992.165'MB/sec zip my value time = 0.81's, unzip length = 0.132'GB zip my value throughput = 162.871'MB/sec zip pipeline throughput = 162.871'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1017.00 avg-zval = 311.55 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1024.00 avg-zval = 311.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6744 value = 3.2643 dict = 1.00 all = 3.1639 } Zip/UnZip{ index = 0.1498 value = 0.3063 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 9.643833 GB avg = 0.181 KB (by entry num) total key len = 1.456353 GB avg = 0.027 KB total ukey len = 1.007197 GB avg = 0.015 KB total ukey num = 0.066327029 Billion total entry num = 0.053134758 Billion write speed all = 0.297243783 MB/sec (with seq num) write speed all = 0.285860928 MB/sec (without seq num) 2024/11/23-19:17:35.978334 139957231949376 [compaction_job.cc:1948] [default] [JOB 34027] Generated table #276828: 129430 keys, 42421781 bytes, temperature: kWarm 2024/11/23-19:17:35.978400 139957231949376 EVENT_LOG_v1 {"time_micros": 1732360655978365, "cf_name": "default", "job": 34027, "event": "table_file_creation", "file_number": 276828, "file_size": 42421781, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237872175, "largest_seqno": 4210213053, "table_properties": {"data_size": 40323632, "index_size": 232704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131630081, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360654, "oldest_key_time": 0, "file_creation_time": 1732360654, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276828, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:17:36.016089 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276829.sst) /dev/shm/ToplingTemp/Topling-163366-LKarNc 2024/11/23-19:17:36.693267 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276829.sst): first pass time = 0.68's, 198.213'MB/sec 2024/11/23-19:17:37.509317 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276829.sst): old prealloc_size = 147639500, real_size = 40898448 2024/11/23-19:17:37.509439 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276829.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.75 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1017.00 avg-zval = 313.99 2024/11/23-19:17:37.509606 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276829.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:17:37.671030 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276829.sst): second pass time = 0.82's, 161.445'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 264.314'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2284.862'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.04's, 975.579'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 47432.112'MB/sec zip my value time = 0.82's, unzip length = 0.132'GB zip my value throughput = 161.445'MB/sec zip pipeline throughput = 161.445'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.75 avg-val = 1017.00 avg-zval = 313.99 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.75 avg-val = 1024.00 avg-zval = 313.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.8477 value = 3.2390 dict = 1.00 all = 3.1410 } Zip/UnZip{ index = 0.1460 value = 0.3087 dict = 1.00 all = 0.3184 } ---------------------------- total value len = 9.776369 GB avg = 0.184 KB (by entry num) total key len = 1.458942 GB avg = 0.027 KB total ukey len = 1.008750 GB avg = 0.015 KB total ukey num = 0.066456459 Billion total entry num = 0.053264188 Billion write speed all = 0.300848119 MB/sec (with seq num) write speed all = 0.289438069 MB/sec (without seq num) 2024/11/23-19:17:37.722647 139957231949376 [compaction_job.cc:1948] [default] [JOB 34027] Generated table #276829: 129430 keys, 42731355 bytes, temperature: kWarm 2024/11/23-19:17:37.722716 139957231949376 EVENT_LOG_v1 {"time_micros": 1732360657722681, "cf_name": "default", "job": 34027, "event": "table_file_creation", "file_number": 276829, "file_size": 42731355, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237885133, "largest_seqno": 4210202941, "table_properties": {"data_size": 40639104, "index_size": 226816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629675, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360656, "oldest_key_time": 0, "file_creation_time": 1732360656, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276829, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:17:37.761042 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276830.sst) /dev/shm/ToplingTemp/Topling-163366-520Rdc 2024/11/23-19:17:38.602145 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276830.sst): first pass time = 0.84's, 159.584'MB/sec 2024/11/23-19:17:39.480326 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276830.sst): old prealloc_size = 147639500, real_size = 40974848 2024/11/23-19:17:39.480452 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276830.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.73 raw-val = 0.1316 GB zip-val = 0.0407 GB avg-val =1017.02 avg-zval = 314.61 2024/11/23-19:17:39.480612 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276830.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:17:39.642100 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276830.sst): second pass time = 0.88's, 150.037'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 264.648'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2175.237'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.272'MB/sec write SST data time = 0.04's, 1005.783'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45141.603'MB/sec zip my value time = 0.88's, unzip length = 0.132'GB zip my value throughput = 150.037'MB/sec zip pipeline throughput = 150.037'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.73 avg-val = 1017.02 avg-zval = 314.61 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.73 avg-val = 1024.02 avg-zval = 314.61 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129427 bytes = 485360 } seq expand size = 905989 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0407 GB dict = 1.38 MB all = 0.0428 GB } UnZip/Zip{ index = 6.9395 value = 3.2327 dict = 1.00 all = 3.1354 } Zip/UnZip{ index = 0.1441 value = 0.3093 dict = 1.00 all = 0.3189 } ---------------------------- total value len = 9.908905 GB avg = 0.186 KB (by entry num) total key len = 1.461530 GB avg = 0.027 KB total ukey len = 1.010303 GB avg = 0.015 KB total ukey num = 0.066585886 Billion total entry num = 0.053393615 Billion write speed all = 0.304450281 MB/sec (with seq num) write speed all = 0.293013110 MB/sec (without seq num) 2024/11/23-19:17:39.693561 139957231949376 [compaction_job.cc:1948] [default] [JOB 34027] Generated table #276830: 129427 keys, 42807379 bytes, temperature: kWarm 2024/11/23-19:17:39.693628 139957231949376 EVENT_LOG_v1 {"time_micros": 1732360659693594, "cf_name": "default", "job": 34027, "event": "table_file_creation", "file_number": 276830, "file_size": 42807379, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868290, "largest_seqno": 4210176199, "table_properties": {"data_size": 40718512, "index_size": 223808, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629968, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360657, "oldest_key_time": 0, "file_creation_time": 1732360657, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276830, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:17:39.731883 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276831.sst) /dev/shm/ToplingTemp/Topling-163366-sjmywa 2024/11/23-19:17:40.397508 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276831.sst): first pass time = 0.67's, 201.654'MB/sec 2024/11/23-19:17:41.261323 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276831.sst): old prealloc_size = 147639500, real_size = 41031264 2024/11/23-19:17:41.261444 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276831.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.71 raw-val = 0.1316 GB zip-val = 0.0408 GB avg-val =1016.99 avg-zval = 315.06 2024/11/23-19:17:41.261597 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276831.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:17:41.423320 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276831.sst): second pass time = 0.86's, 152.687'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 264.631'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2223.431'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.274'MB/sec write SST data time = 0.04's, 977.955'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 48321.048'MB/sec zip my value time = 0.86's, unzip length = 0.132'GB zip my value throughput = 152.687'MB/sec zip pipeline throughput = 152.687'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.71 avg-val = 1016.99 avg-zval = 315.06 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.71 avg-val = 1023.99 avg-zval = 315.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129431 bytes = 485376 } seq expand size = 906017 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0408 GB dict = 1.38 MB all = 0.0429 GB } UnZip/Zip{ index = 7.0343 value = 3.2280 dict = 1.00 all = 3.1313 } Zip/UnZip{ index = 0.1422 value = 0.3098 dict = 1.00 all = 0.3194 } ---------------------------- total value len = 10.041440 GB avg = 0.188 KB (by entry num) total key len = 1.464119 GB avg = 0.027 KB total ukey len = 1.011857 GB avg = 0.015 KB total ukey num = 0.066715317 Billion total entry num = 0.053523046 Billion write speed all = 0.308053627 MB/sec (with seq num) write speed all = 0.296589278 MB/sec (without seq num) 2024/11/23-19:17:41.475695 139957231949376 [compaction_job.cc:1948] [default] [JOB 34027] Generated table #276831: 129431 keys, 42863938 bytes, temperature: kWarm 2024/11/23-19:17:41.475757 139957231949376 EVENT_LOG_v1 {"time_micros": 1732360661475727, "cf_name": "default", "job": 34027, "event": "table_file_creation", "file_number": 276831, "file_size": 42863938, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237874784, "largest_seqno": 4210214264, "table_properties": {"data_size": 40777936, "index_size": 220800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629780, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360659, "oldest_key_time": 0, "file_creation_time": 1732360659, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276831, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:17:41.513815 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276832.sst) /dev/shm/ToplingTemp/Topling-163366-rtiufc 2024/11/23-19:17:42.402473 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276832.sst): first pass time = 0.89's, 151.042'MB/sec 2024/11/23-19:17:43.216623 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276832.sst): old prealloc_size = 147639500, real_size = 40706944 2024/11/23-19:17:43.216748 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276832.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129433 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.78 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1016.97 avg-zval = 312.47 2024/11/23-19:17:43.216895 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276832.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:17:43.378279 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276832.sst): second pass time = 0.81's, 161.843'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 263.184'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2249.668'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.272'MB/sec write SST data time = 0.04's, 1004.300'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45485.546'MB/sec zip my value time = 0.81's, unzip length = 0.132'GB zip my value throughput = 161.843'MB/sec zip pipeline throughput = 161.843'MB/sec entries = 129433 avg-key = 20.00 avg-zkey = 1.78 avg-val = 1016.97 avg-zval = 312.47 usrkeys = 129433 avg-key = 12.00 avg-zkey = 1.78 avg-val = 1023.97 avg-zval = 312.47 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129433 bytes = 485392 } seq expand size = 906031 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0425 GB } UnZip/Zip{ index = 6.7376 value = 3.2546 dict = 1.00 all = 3.1551 } Zip/UnZip{ index = 0.1484 value = 0.3073 dict = 1.00 all = 0.3169 } ---------------------------- total value len = 10.173976 GB avg = 0.190 KB (by entry num) total key len = 1.466707 GB avg = 0.027 KB total ukey len = 1.013410 GB avg = 0.015 KB total ukey num = 0.066844750 Billion total entry num = 0.053652479 Billion write speed all = 0.311655172 MB/sec (with seq num) write speed all = 0.300163701 MB/sec (without seq num) 2024/11/23-19:17:43.430069 139957231949376 [compaction_job.cc:1948] [default] [JOB 34027] Generated table #276832: 129433 keys, 42540030 bytes, temperature: kWarm 2024/11/23-19:17:43.430135 139957231949376 EVENT_LOG_v1 {"time_micros": 1732360663430100, "cf_name": "default", "job": 34027, "event": "table_file_creation", "file_number": 276832, "file_size": 42540030, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237874127, "largest_seqno": 4210214265, "table_properties": {"data_size": 40443888, "index_size": 230528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588660, "raw_average_key_size": 20, "raw_value_size": 131629454, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129433, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360661, "oldest_key_time": 0, "file_creation_time": 1732360661, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276832, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:17:43.467185 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276833.sst) /dev/shm/ToplingTemp/Topling-163366-AllyUc 2024/11/23-19:17:44.033377 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276833.sst): first pass time = 0.57's, 237.074'MB/sec 2024/11/23-19:17:44.777402 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276833.sst): old prealloc_size = 147639500, real_size = 40579936 2024/11/23-19:17:44.777542 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276833.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.00 avg-zval = 311.48 2024/11/23-19:17:44.777728 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276833.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:17:44.959474 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276833.sst): second pass time = 0.74's, 177.211'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 232.964'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2025.997'MB/sec (index lex order gen) rebuild zvType time = 0.13's, 0.248'MB/sec write SST data time = 0.05's, 825.852'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 35103.243'MB/sec zip my value time = 0.74's, unzip length = 0.132'GB zip my value throughput = 177.211'MB/sec zip pipeline throughput = 177.211'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1017.00 avg-zval = 311.48 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1024.00 avg-zval = 311.48 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129429 bytes = 485376 } seq expand size = 906003 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6744 value = 3.2651 dict = 1.00 all = 3.1646 } Zip/UnZip{ index = 0.1498 value = 0.3063 dict = 1.00 all = 0.3160 } ---------------------------- total value len = 10.306512 GB avg = 0.192 KB (by entry num) total key len = 1.469296 GB avg = 0.027 KB total ukey len = 1.014963 GB avg = 0.015 KB total ukey num = 0.066974179 Billion total entry num = 0.053781908 Billion write speed all = 0.315259501 MB/sec (with seq num) write speed all = 0.303740795 MB/sec (without seq num) 2024/11/23-19:17:45.012211 139957231949376 [compaction_job.cc:1948] [default] [JOB 34027] Generated table #276833: 129429 keys, 42412735 bytes, temperature: kWarm 2024/11/23-19:17:45.012282 139957231949376 EVENT_LOG_v1 {"time_micros": 1732360665012246, "cf_name": "default", "job": 34027, "event": "table_file_creation", "file_number": 276833, "file_size": 42412735, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237873605, "largest_seqno": 4210211519, "table_properties": {"data_size": 40314704, "index_size": 232704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629751, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360663, "oldest_key_time": 0, "file_creation_time": 1732360663, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276833, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:17:45.050624 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276834.sst) /dev/shm/ToplingTemp/Topling-163366-yMj5Sc 2024/11/23-19:17:45.658409 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276834.sst): first pass time = 0.61's, 220.847'MB/sec 2024/11/23-19:17:46.602473 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276834.sst): old prealloc_size = 147639500, real_size = 40585824 2024/11/23-19:17:46.602599 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276834.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.00 avg-zval = 311.53 2024/11/23-19:17:46.602766 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276834.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:17:46.767217 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276834.sst): second pass time = 0.94's, 139.543'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 257.526'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2188.919'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.267'MB/sec write SST data time = 0.04's, 973.126'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44825.357'MB/sec zip my value time = 0.94's, unzip length = 0.132'GB zip my value throughput = 139.543'MB/sec zip pipeline throughput = 172.657'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1017.00 avg-zval = 311.53 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1024.00 avg-zval = 311.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129429 bytes = 485376 } seq expand size = 906003 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6744 value = 3.2646 dict = 1.00 all = 3.1642 } Zip/UnZip{ index = 0.1498 value = 0.3063 dict = 1.00 all = 0.3160 } ---------------------------- total value len = 10.439048 GB avg = 0.194 KB (by entry num) total key len = 1.471885 GB avg = 0.027 KB total ukey len = 1.016516 GB avg = 0.015 KB total ukey num = 0.067103608 Billion total entry num = 0.053911337 Billion write speed all = 0.318861594 MB/sec (with seq num) write speed all = 0.307315727 MB/sec (without seq num) 2024/11/23-19:17:46.819227 139957231949376 [compaction_job.cc:1948] [default] [JOB 34027] Generated table #276834: 129429 keys, 42418372 bytes, temperature: kWarm 2024/11/23-19:17:46.819295 139957231949376 EVENT_LOG_v1 {"time_micros": 1732360666819260, "cf_name": "default", "job": 34027, "event": "table_file_creation", "file_number": 276834, "file_size": 42418372, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237872082, "largest_seqno": 4210212344, "table_properties": {"data_size": 40320592, "index_size": 232704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629898, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360665, "oldest_key_time": 0, "file_creation_time": 1732360665, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276834, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:17:46.856635 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276835.sst) /dev/shm/ToplingTemp/Topling-163366-NfYoLb 2024/11/23-19:17:47.541414 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276835.sst): first pass time = 0.68's, 196.013'MB/sec 2024/11/23-19:17:48.421041 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276835.sst): old prealloc_size = 147639500, real_size = 41077120 2024/11/23-19:17:48.421164 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276835.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0408 GB avg-val =1017.00 avg-zval = 315.40 2024/11/23-19:17:48.421348 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276835.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:17:48.587016 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276835.sst): second pass time = 0.88's, 149.776'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 258.594'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2270.487'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.268'MB/sec write SST data time = 0.05's, 947.299'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 49734.358'MB/sec zip my value time = 0.88's, unzip length = 0.132'GB zip my value throughput = 149.776'MB/sec zip pipeline throughput = 149.776'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.00 avg-zval = 315.40 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1024.00 avg-zval = 315.40 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129429 bytes = 485376 } seq expand size = 906003 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0408 GB dict = 1.38 MB all = 0.0429 GB } UnZip/Zip{ index = 6.9896 value = 3.2244 dict = 1.00 all = 3.1279 } Zip/UnZip{ index = 0.1431 value = 0.3101 dict = 1.00 all = 0.3197 } ---------------------------- total value len = 10.571583 GB avg = 0.196 KB (by entry num) total key len = 1.474473 GB avg = 0.027 KB total ukey len = 1.018069 GB avg = 0.015 KB total ukey num = 0.067233037 Billion total entry num = 0.054040766 Billion write speed all = 0.322463226 MB/sec (with seq num) write speed all = 0.310890204 MB/sec (without seq num) 2024/11/23-19:17:48.639186 139957231949376 [compaction_job.cc:1948] [default] [JOB 34027] Generated table #276835: 129429 keys, 42910241 bytes, temperature: kWarm 2024/11/23-19:17:48.639266 139957231949376 EVENT_LOG_v1 {"time_micros": 1732360668639229, "cf_name": "default", "job": 34027, "event": "table_file_creation", "file_number": 276835, "file_size": 42910241, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237916889, "largest_seqno": 4210203119, "table_properties": {"data_size": 40822384, "index_size": 222208, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629600, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360666, "oldest_key_time": 0, "file_creation_time": 1732360666, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276835, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:17:48.676794 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276836.sst) /dev/shm/ToplingTemp/Topling-163366-R3KtYc 2024/11/23-19:17:49.406166 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276836.sst): first pass time = 0.73's, 184.031'MB/sec 2024/11/23-19:17:50.365575 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276836.sst): old prealloc_size = 147639500, real_size = 40590752 2024/11/23-19:17:50.365697 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276836.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.02 avg-zval = 311.57 2024/11/23-19:17:50.365852 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276836.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:17:50.524862 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276836.sst): second pass time = 0.96's, 137.307'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 266.372'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2285.961'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.04's, 1017.630'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 52436.615'MB/sec zip my value time = 0.96's, unzip length = 0.132'GB zip my value throughput = 137.307'MB/sec zip pipeline throughput = 193.537'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1017.02 avg-zval = 311.57 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1024.02 avg-zval = 311.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129427 bytes = 485360 } seq expand size = 905989 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6742 value = 3.2642 dict = 1.00 all = 3.1638 } Zip/UnZip{ index = 0.1498 value = 0.3064 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 10.704119 GB avg = 0.198 KB (by entry num) total key len = 1.477062 GB avg = 0.027 KB total ukey len = 1.019622 GB avg = 0.015 KB total ukey num = 0.067362464 Billion total entry num = 0.054170193 Billion write speed all = 0.326063488 MB/sec (with seq num) write speed all = 0.314463351 MB/sec (without seq num) 2024/11/23-19:17:50.576401 139957231949376 [compaction_job.cc:1948] [default] [JOB 34027] Generated table #276836: 129427 keys, 42423683 bytes, temperature: kWarm 2024/11/23-19:17:50.576467 139957231949376 EVENT_LOG_v1 {"time_micros": 1732360670576433, "cf_name": "default", "job": 34027, "event": "table_file_creation", "file_number": 276836, "file_size": 42423683, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237871365, "largest_seqno": 4210209779, "table_properties": {"data_size": 40325520, "index_size": 232704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131630080, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360668, "oldest_key_time": 0, "file_creation_time": 1732360668, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276836, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:17:50.614252 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276837.sst) /dev/shm/ToplingTemp/Topling-163366-4iHxr9 2024/11/23-19:17:51.193285 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276837.sst): first pass time = 0.58's, 231.811'MB/sec 2024/11/23-19:17:51.978168 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276837.sst): old prealloc_size = 147639500, real_size = 40478464 2024/11/23-19:17:51.978302 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276837.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1017.00 avg-zval = 310.78 2024/11/23-19:17:51.978398 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276837.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:17:52.140852 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276837.sst): second pass time = 0.78's, 167.859'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 258.125'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2180.669'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.04's, 937.010'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 29222.321'MB/sec zip my value time = 0.78's, unzip length = 0.132'GB zip my value throughput = 167.859'MB/sec zip pipeline throughput = 167.859'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.00 avg-zval = 310.78 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1019.03 avg-zval = 310.78 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 37610 bytes = 141056 } seq expand size = 263270 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0420 GB } UnZip/Zip{ index = 6.9897 value = 3.2724 dict = 1.00 all = 3.1967 } Zip/UnZip{ index = 0.1431 value = 0.3056 dict = 1.00 all = 0.3128 } ---------------------------- total value len = 10.836012 GB avg = 0.200 KB (by entry num) total key len = 1.479650 GB avg = 0.027 KB total ukey len = 1.021176 GB avg = 0.015 KB total ukey num = 0.067491894 Billion total entry num = 0.054299623 Billion write speed all = 0.329649005 MB/sec (with seq num) write speed all = 0.318021653 MB/sec (without seq num) 2024/11/23-19:17:52.193814 139957231949376 [compaction_job.cc:1948] [default] [JOB 34027] Generated table #276837: 129430 keys, 41986989 bytes, temperature: kWarm 2024/11/23-19:17:52.193886 139957231949376 EVENT_LOG_v1 {"time_micros": 1732360672193848, "cf_name": "default", "job": 34027, "event": "table_file_creation", "file_number": 276837, "file_size": 41986989, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4210210070, "table_properties": {"data_size": 40223728, "index_size": 222208, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629667, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360670, "oldest_key_time": 0, "file_creation_time": 1732360670, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276837, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:17:52.232963 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276838.sst) /dev/shm/ToplingTemp/Topling-163366-6xDSMa 2024/11/23-19:17:52.807400 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276838.sst): first pass time = 0.57's, 233.697'MB/sec 2024/11/23-19:17:53.641152 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276838.sst): old prealloc_size = 147639500, real_size = 41072528 2024/11/23-19:17:53.641286 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276838.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.71 raw-val = 0.1316 GB zip-val = 0.0408 GB avg-val =1016.99 avg-zval = 315.37 2024/11/23-19:17:53.641442 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276838.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:17:53.859959 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276838.sst): second pass time = 0.83's, 158.196'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.22's, 196.026'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1987.124'MB/sec (index lex order gen) rebuild zvType time = 0.16's, 0.209'MB/sec write SST data time = 0.06's, 674.388'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45508.518'MB/sec zip my value time = 0.83's, unzip length = 0.132'GB zip my value throughput = 158.196'MB/sec zip pipeline throughput = 158.196'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.71 avg-val = 1016.99 avg-zval = 315.37 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.71 avg-val = 1023.43 avg-zval = 315.37 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 119249 bytes = 447200 } seq expand size = 834743 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0408 GB dict = 1.38 MB all = 0.0429 GB } UnZip/Zip{ index = 7.0282 value = 3.2247 dict = 1.00 all = 3.1295 } Zip/UnZip{ index = 0.1423 value = 0.3101 dict = 1.00 all = 0.3195 } ---------------------------- total value len = 10.968476 GB avg = 0.202 KB (by entry num) total key len = 1.482239 GB avg = 0.027 KB total ukey len = 1.022729 GB avg = 0.015 KB total ukey num = 0.067621325 Billion total entry num = 0.054429054 Billion write speed all = 0.333248581 MB/sec (with seq num) write speed all = 0.321594051 MB/sec (without seq num) 2024/11/23-19:17:53.924013 139957231949376 [compaction_job.cc:1948] [default] [JOB 34027] Generated table #276838: 129431 keys, 42887801 bytes, temperature: kWarm 2024/11/23-19:17:53.924103 139957231949376 EVENT_LOG_v1 {"time_micros": 1732360673924057, "cf_name": "default", "job": 34027, "event": "table_file_creation", "file_number": 276838, "file_size": 42887801, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4210207581, "table_properties": {"data_size": 40819008, "index_size": 220992, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629432, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360672, "oldest_key_time": 0, "file_creation_time": 1732360672, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276838, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:17:53.997126 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276839.sst) /dev/shm/ToplingTemp/Topling-163366-IS1e98 2024/11/23-19:17:54.673667 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276839.sst): first pass time = 0.68's, 198.403'MB/sec 2024/11/23-19:17:55.508788 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276839.sst): old prealloc_size = 147639500, real_size = 40631232 2024/11/23-19:17:55.508909 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276839.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.75 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1017.01 avg-zval = 311.93 2024/11/23-19:17:55.509044 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276839.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:17:55.671941 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276839.sst): second pass time = 0.83's, 157.916'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 259.252'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2256.560'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.269'MB/sec write SST data time = 0.04's, 984.498'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 49669.466'MB/sec zip my value time = 0.83's, unzip length = 0.132'GB zip my value throughput = 157.916'MB/sec zip pipeline throughput = 157.916'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.75 avg-val = 1017.01 avg-zval = 311.93 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.75 avg-val = 1021.25 avg-zval = 311.93 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 78520 bytes = 294464 } seq expand size = 549640 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0423 GB } UnZip/Zip{ index = 6.8748 value = 3.2604 dict = 1.00 all = 3.1735 } Zip/UnZip{ index = 0.1455 value = 0.3067 dict = 1.00 all = 0.3151 } ---------------------------- total value len = 11.100656 GB avg = 0.203 KB (by entry num) total key len = 1.484827 GB avg = 0.027 KB total ukey len = 1.024282 GB avg = 0.015 KB total ukey num = 0.067750754 Billion total entry num = 0.054558483 Billion write speed all = 0.336839371 MB/sec (with seq num) write speed all = 0.325157693 MB/sec (without seq num) 2024/11/23-19:17:55.723807 139957231949376 [compaction_job.cc:1948] [default] [JOB 34027] Generated table #276839: 129429 keys, 42293222 bytes, temperature: kWarm 2024/11/23-19:17:55.723887 139957231949376 EVENT_LOG_v1 {"time_micros": 1732360675723853, "cf_name": "default", "job": 34027, "event": "table_file_creation", "file_number": 276839, "file_size": 42293222, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4210194489, "table_properties": {"data_size": 40372784, "index_size": 225920, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629946, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360673, "oldest_key_time": 0, "file_creation_time": 1732360673, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276839, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:17:55.774700 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276840.sst) /dev/shm/ToplingTemp/Topling-163366-zS4tCb 2024/11/23-19:17:56.304816 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276840.sst): first pass time = 0.53's, 253.204'MB/sec 2024/11/23-19:17:57.128902 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276840.sst): old prealloc_size = 147639500, real_size = 41029056 2024/11/23-19:17:57.129060 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276840.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129425 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0408 GB avg-val =1017.03 avg-zval = 315.04 2024/11/23-19:17:57.129243 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276840.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:17:57.342408 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276840.sst): second pass time = 0.82's, 159.879'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.21's, 200.733'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1681.595'MB/sec (index lex order gen) rebuild zvType time = 0.15's, 0.222'MB/sec write SST data time = 0.07's, 634.031'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 41548.106'MB/sec zip my value time = 0.82's, unzip length = 0.132'GB zip my value throughput = 159.879'MB/sec zip pipeline throughput = 159.879'MB/sec entries = 129425 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.03 avg-zval = 315.04 usrkeys = 129425 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1023.58 avg-zval = 315.04 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 121111 bytes = 454176 } seq expand size = 847777 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0408 GB dict = 1.38 MB all = 0.0429 GB } UnZip/Zip{ index = 6.9613 value = 3.2283 dict = 1.00 all = 3.1322 } Zip/UnZip{ index = 0.1437 value = 0.3098 dict = 1.00 all = 0.3193 } ---------------------------- total value len = 11.233133 GB avg = 0.205 KB (by entry num) total key len = 1.487416 GB avg = 0.027 KB total ukey len = 1.025835 GB avg = 0.015 KB total ukey num = 0.067880179 Billion total entry num = 0.054687908 Billion write speed all = 0.340439064 MB/sec (with seq num) write speed all = 0.328730198 MB/sec (without seq num) 2024/11/23-19:17:57.397006 139957231949376 [compaction_job.cc:1948] [default] [JOB 34027] Generated table #276840: 129425 keys, 42850656 bytes, temperature: kWarm 2024/11/23-19:17:57.397098 139957231949376 EVENT_LOG_v1 {"time_micros": 1732360677397049, "cf_name": "default", "job": 34027, "event": "table_file_creation", "file_number": 276840, "file_size": 42850656, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4210204930, "table_properties": {"data_size": 40773424, "index_size": 223104, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588500, "raw_average_key_size": 20, "raw_value_size": 131629391, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129425, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360675, "oldest_key_time": 0, "file_creation_time": 1732360675, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276840, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:17:57.447679 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276841.sst) /dev/shm/ToplingTemp/Topling-163366-MMJTCb 2024/11/23-19:17:57.968764 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276841.sst): first pass time = 0.52's, 257.610'MB/sec 2024/11/23-19:17:58.754292 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276841.sst): old prealloc_size = 147639500, real_size = 40531536 2024/11/23-19:17:58.754435 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276841.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.74 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.02 avg-zval = 311.17 2024/11/23-19:17:58.754545 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276841.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:17:58.915791 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276841.sst): second pass time = 0.78's, 167.750'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 260.945'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2248.478'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.277'MB/sec write SST data time = 0.04's, 944.879'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 48234.905'MB/sec zip my value time = 0.78's, unzip length = 0.132'GB zip my value throughput = 167.750'MB/sec zip pipeline throughput = 167.750'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.74 avg-val = 1017.02 avg-zval = 311.17 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.74 avg-val = 1020.45 avg-zval = 311.17 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 63493 bytes = 238112 } seq expand size = 444451 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0421 GB } UnZip/Zip{ index = 6.8825 value = 3.2684 dict = 1.00 all = 3.1852 } Zip/UnZip{ index = 0.1453 value = 0.3060 dict = 1.00 all = 0.3139 } ---------------------------- total value len = 11.365207 GB avg = 0.207 KB (by entry num) total key len = 1.490004 GB avg = 0.027 KB total ukey len = 1.027388 GB avg = 0.015 KB total ukey num = 0.068009606 Billion total entry num = 0.054817335 Billion write speed all = 0.344028536 MB/sec (with seq num) write speed all = 0.332292454 MB/sec (without seq num) 2024/11/23-19:17:58.973796 139957231949376 [compaction_job.cc:1948] [default] [JOB 34027] Generated table #276841: 129427 keys, 42137382 bytes, temperature: kWarm 2024/11/23-19:17:58.973862 139957231949376 EVENT_LOG_v1 {"time_micros": 1732360678973829, "cf_name": "default", "job": 34027, "event": "table_file_creation", "file_number": 276841, "file_size": 42137382, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4210210100, "table_properties": {"data_size": 40273344, "index_size": 225664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629395, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360677, "oldest_key_time": 0, "file_creation_time": 1732360677, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276841, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:17:59.012217 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276842.sst) /dev/shm/ToplingTemp/Topling-163366-y5T9f9 2024/11/23-19:17:59.595416 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276842.sst): first pass time = 0.58's, 230.159'MB/sec 2024/11/23-19:18:00.402156 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276842.sst): old prealloc_size = 147639500, real_size = 40630144 2024/11/23-19:18:00.402295 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276842.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.73 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1017.01 avg-zval = 311.94 2024/11/23-19:18:00.402390 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276842.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:18:00.562192 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276842.sst): second pass time = 0.81's, 163.314'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 263.525'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1965.966'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.277'MB/sec write SST data time = 0.04's, 980.182'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45492.689'MB/sec zip my value time = 0.81's, unzip length = 0.132'GB zip my value throughput = 163.314'MB/sec zip pipeline throughput = 163.314'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.73 avg-val = 1017.01 avg-zval = 311.94 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.73 avg-val = 1019.47 avg-zval = 311.94 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 45610 bytes = 171056 } seq expand size = 319270 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0422 GB } UnZip/Zip{ index = 6.9475 value = 3.2602 dict = 1.00 all = 3.1828 } Zip/UnZip{ index = 0.1439 value = 0.3067 dict = 1.00 all = 0.3142 } ---------------------------- total value len = 11.497156 GB avg = 0.209 KB (by entry num) total key len = 1.492593 GB avg = 0.027 KB total ukey len = 1.028941 GB avg = 0.015 KB total ukey num = 0.068139034 Billion total entry num = 0.054946763 Billion write speed all = 0.347613675 MB/sec (with seq num) write speed all = 0.335850401 MB/sec (without seq num) 2024/11/23-19:18:00.620660 139957231949376 [compaction_job.cc:1948] [default] [JOB 34027] Generated table #276842: 129428 keys, 42169203 bytes, temperature: kWarm 2024/11/23-19:18:00.620732 139957231949376 EVENT_LOG_v1 {"time_micros": 1732360680620696, "cf_name": "default", "job": 34027, "event": "table_file_creation", "file_number": 276842, "file_size": 42169203, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4210199943, "table_properties": {"data_size": 40374064, "index_size": 223552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629278, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360678, "oldest_key_time": 0, "file_creation_time": 1732360678, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276842, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:18:00.657001 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276843.sst) /dev/shm/ToplingTemp/Topling-163366-XgVy7a 2024/11/23-19:18:01.390014 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276843.sst): first pass time = 0.73's, 183.117'MB/sec 2024/11/23-19:18:02.172194 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276843.sst): old prealloc_size = 147639500, real_size = 40479280 2024/11/23-19:18:02.172340 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276843.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129432 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1016.98 avg-zval = 310.77 2024/11/23-19:18:02.172433 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276843.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:18:02.337639 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276843.sst): second pass time = 0.78's, 168.450'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 253.825'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2072.896'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.266'MB/sec write SST data time = 0.04's, 964.115'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45165.602'MB/sec zip my value time = 0.78's, unzip length = 0.132'GB zip my value throughput = 168.450'MB/sec zip pipeline throughput = 168.450'MB/sec entries = 129432 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1016.98 avg-zval = 310.77 usrkeys = 129432 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1019.01 avg-zval = 310.77 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 37467 bytes = 140512 } seq expand size = 262269 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0420 GB } UnZip/Zip{ index = 6.9757 value = 3.2724 dict = 1.00 all = 3.1966 } Zip/UnZip{ index = 0.1434 value = 0.3056 dict = 1.00 all = 0.3128 } ---------------------------- total value len = 11.629048 GB avg = 0.211 KB (by entry num) total key len = 1.495182 GB avg = 0.027 KB total ukey len = 1.030494 GB avg = 0.015 KB total ukey num = 0.068268466 Billion total entry num = 0.055076195 Billion write speed all = 0.351195784 MB/sec (with seq num) write speed all = 0.339405360 MB/sec (without seq num) 2024/11/23-19:18:02.394159 139957231949376 [compaction_job.cc:1948] [default] [JOB 34027] Generated table #276843: 129432 keys, 41987673 bytes, temperature: kWarm 2024/11/23-19:18:02.394237 139957231949376 EVENT_LOG_v1 {"time_micros": 1732360682394194, "cf_name": "default", "job": 34027, "event": "table_file_creation", "file_number": 276843, "file_size": 41987673, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4210209701, "table_properties": {"data_size": 40224096, "index_size": 222656, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588640, "raw_average_key_size": 20, "raw_value_size": 131630104, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360680, "oldest_key_time": 0, "file_creation_time": 1732360680, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276843, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:18:02.432709 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276844.sst) /dev/shm/ToplingTemp/Topling-163366-sE4BIb 2024/11/23-19:18:02.992019 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276844.sst): first pass time = 0.56's, 239.990'MB/sec 2024/11/23-19:18:03.763710 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276844.sst): old prealloc_size = 147639500, real_size = 40467024 2024/11/23-19:18:03.763839 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276844.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1017.00 avg-zval = 310.69 2024/11/23-19:18:03.763943 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276844.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:18:03.927569 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276844.sst): second pass time = 0.77's, 170.723'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 256.080'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2111.067'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.269'MB/sec write SST data time = 0.04's, 966.478'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 47063.527'MB/sec zip my value time = 0.77's, unzip length = 0.132'GB zip my value throughput = 170.723'MB/sec zip pipeline throughput = 170.723'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.00 avg-zval = 310.69 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1018.75 avg-zval = 310.69 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 32313 bytes = 121184 } seq expand size = 226191 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0420 GB } UnZip/Zip{ index = 6.9936 value = 3.2734 dict = 1.00 all = 3.1990 } Zip/UnZip{ index = 0.1430 value = 0.3055 dict = 1.00 all = 0.3126 } ---------------------------- total value len = 11.760904 GB avg = 0.213 KB (by entry num) total key len = 1.497770 GB avg = 0.027 KB total ukey len = 1.032048 GB avg = 0.015 KB total ukey num = 0.068397895 Billion total entry num = 0.055205624 Billion write speed all = 0.354778332 MB/sec (with seq num) write speed all = 0.342960704 MB/sec (without seq num) 2024/11/23-19:18:03.979671 139957231949376 [compaction_job.cc:1948] [default] [JOB 34027] Generated table #276844: 129429 keys, 41955892 bytes, temperature: kWarm 2024/11/23-19:18:03.979743 139957231949376 EVENT_LOG_v1 {"time_micros": 1732360683979706, "cf_name": "default", "job": 34027, "event": "table_file_creation", "file_number": 276844, "file_size": 41955892, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4210205851, "table_properties": {"data_size": 40212416, "index_size": 222080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629562, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360682, "oldest_key_time": 0, "file_creation_time": 1732360682, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276844, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:18:04.022462 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276845.sst) /dev/shm/ToplingTemp/Topling-163366-Im9NLa 2024/11/23-19:18:04.033776 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276845.sst): first pass time = 0.01's, 240.446'MB/sec 2024/11/23-19:18:04.064968 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276845.sst): old prealloc_size = 147639500, real_size = 878848 2024/11/23-19:18:04.065062 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276845.sst): index type = FixedLenKeyIndex , store type = DictZipBlobStore usrkeys = 2612 prefix = 4 raw-key = 0.0000 GB zip-key = 0.0000 GB avg-key = 8.00 avg-zkey = 9.04 raw-val = 0.0027 GB zip-val = 0.0009 GB avg-val =1017.09 avg-zval = 327.10 2024/11/23-19:18:04.068496 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00/276845.sst): second pass time = 0.03's, 87.066'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.00's, 261.453'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 524.590'MB/sec (index lex order gen) rebuild zvType time = 0.00's, 0.293'MB/sec write SST data time = 0.00's, 737.610'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.033'MB, throughput = 1377.331'MB/sec zip my value time = 0.03's, unzip length = 0.003'GB zip my value throughput = 87.066'MB/sec zip pipeline throughput = 87.066'MB/sec entries = 2612 avg-key = 20.00 avg-zkey = 9.04 avg-val = 1017.09 avg-zval = 327.10 usrkeys = 2612 avg-key = 12.00 avg-zkey = 9.04 avg-val = 1021.90 avg-zval = 327.10 TagRS{ kinds = RS_Key0_TagN bytes = 520 } TagArray{ size = 1794 bytes = 6736 } seq expand size = 12558 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0000 GB value = 0.0027 GB dict = 0.03 MB all = 0.0027 GB } __ZipSize{ index = 0.0000 GB value = 0.0009 GB dict = 0.03 MB all = 0.0009 GB } UnZip/Zip{ index = 1.3272 value = 3.1094 dict = 1.00 all = 2.9470 } Zip/UnZip{ index = 0.7534 value = 0.3216 dict = 1.00 all = 0.3393 } ---------------------------- total value len = 11.763573 GB avg = 0.213 KB (by entry num) total key len = 1.497823 GB avg = 0.027 KB total ukey len = 1.032079 GB avg = 0.015 KB total ukey num = 0.068400507 Billion total entry num = 0.055208236 Billion write speed all = 0.354849814 MB/sec (with seq num) write speed all = 0.343031672 MB/sec (without seq num) 2024/11/23-19:18:04.075184 139957231949376 [compaction_job.cc:1948] [default] [JOB 34027] Generated table #276845: 2612 keys, 919194 bytes, temperature: kWarm 2024/11/23-19:18:04.075251 139957231949376 EVENT_LOG_v1 {"time_micros": 1732360684075215, "cf_name": "default", "job": 34027, "event": "table_file_creation", "file_number": 276845, "file_size": 919194, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4210213491, "table_properties": {"data_size": 854384, "index_size": 23616, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 52240, "raw_average_key_size": 20, "raw_value_size": 2656634, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 2612, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-FixedLenKeyIndex-DictZipBlobStore", "creation_time": 1732360684, "oldest_key_time": 0, "file_creation_time": 1732360684, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276845, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:18:04.084518 139957231949376 [compaction_job.cc:703] job-34027: subcompact[0], size: 0.935387 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34027/att-00 [276823,276824,276825,276826,276827,276828,276829,276830,276831,276832,276833,276834,276835,276836,276837,276838,276839,276840,276841,276842,276843,276844,276845] 2024/11/23-19:18:04.223350 139957231949376 (Original Log Time 2024/11/23-19:17:25.364747) EVENT_LOG_v1 {"time_micros": 1732360645364708, "job": 34027, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [276789], "files_L3": [276716, 276717, 276718, 276719, 276720, 276721, 276722, 276723, 276724, 276725, 276726, 276727, 276728, 276729, 276730, 276731, 276732, 276733, 276734, 276735, 276736, 276819], "score": 0.994195, "input_data_size": 967178115, "oldest_snapshot_seqno": 4210371669} 2024/11/23-19:18:04.223364 139957231949376 (Original Log Time 2024/11/23-19:18:04.221697) [compaction_job.cc:2022] [default] [JOB 34027] Compacted 1@2 + 22@3 files to L3 => 935387456 bytes 2024/11/23-19:18:04.223367 139957231949376 (Original Log Time 2024/11/23-19:18:04.223243) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 23 0 0 0] max score 0.03, MB/sec: 25.0 rd, 24.2 wr, level 3, files in(1, 22) out(23 +0 blob) MB in(31.2, 891.2 +0.0 blob) out(892.1 +0.0 blob), read-write-amplify(58.2) write-amplify(28.6) OK, records in: 2879375, records dropped: 29303 output_compression: Snappy 2024/11/23-19:18:04.223375 139957231949376 (Original Log Time 2024/11/23-19:18:04.223317) EVENT_LOG_v1 {"time_micros": 1732360684223283, "cf": "default", "job": 34027, "event": "compaction_finished", "compaction_time_micros": 38719777, "compaction_time_cpu_micros": 8241946, "output_level": 3, "num_output_files": 23, "total_output_size": 935387456, "num_input_records": 2879375, "num_output_records": 2850072, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 657622041, "file_range_sync_nanos": 0, "file_fsync_nanos": 16902, "file_prepare_write_nanos": 49182, "lsm_state": [0, 0, 0, 23, 0, 0, 0]} 2024/11/23-19:18:04.356967 139957231949376 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732360645,"executesMs":39004,"compactionJobId":34027,"attempt":0,"compactionInputRawBytes":2985896943,"compactionInputZipBytes":967178115,"compactionOutputRawBytes":0,"compactionOutputZipBytes":935387456,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-d3a5f640c78c3b54889bfd978ffeca60-3dca488abf478c1c-00"}}, response = 2024/11/23-19:18:04.357074 139957231949376 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-34027/att-00: olev 3, work 39.004 s, result 7.763 ms, install 1.697 ms, input{raw 2.781 GiB zip 922.373 MiB}