2024/11/22-18:34:52.807483 140025196639808 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00: fork to child time = 0.009759 sec 2024/11/22-18:34:52.808366 140025196639808 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00 2024/11/22-18:34:52.816994 140025196639808 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00 2024/11/22-18:34:52.818161 140025196639808 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-18:34:52.818419 140025196639808 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-18:34:52.870177 140025196639808 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-07647/att-00/MANIFEST-206347 2024/11/22-18:34:52.902580 140025196639808 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-07647/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 220745, last_sequence is 3468914679, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 220605 2024/11/22-18:34:52.902588 140025196639808 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-18:34:52.902707 140025196639808 [version_set.cc:5721] Creating manifest 220745 2024/11/22-18:34:52.933638 140025196639808 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-18:34:52.956225 140025196639808 [compaction_job.cc:2437] [default] [JOB 7647] Compacting 2@2 + 15@3 files to L3, score 0.95, subcompactions 1 : 1 2024/11/22-18:34:52.956246 140025196639808 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 5810 Base level 2, inputs: [220734(1556KB) 220739(27MB)], [220703(36MB) 220704(36MB) 220705(35MB) 220706(34MB) 220707(36MB) 220708(36MB) 220709(35MB) 220710(36MB) 220711(8701KB) 220681(36MB) 220682(35MB) 220683(36MB) 220684(35MB) 220685(35MB) 220686(36MB)] 2024/11/22-18:34:52.959511 140025196639808 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-18:34:52.959543 140025196639808 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-18:34:52.959562 140025196639808 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-18:34:52.959567 140025196639808 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-18:34:52.959577 140025196639808 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-18:34:53.321772 140025196639808 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220746.sst) /dev/shm/ToplingTemp/Topling-241942-YllHCI 2024/11/22-18:34:54.542053 140025196639808 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220746.sst): first pass time = 1.22's, 109.994'MB/sec 2024/11/22-18:34:56.918061 140025196639808 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220746.sst): old prealloc_size = 147639500, real_size = 33735968 2024/11/22-18:34:56.918627 140025196639808 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220746.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923526 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0317 GB avg-val = 117.33 avg-zval = 34.38 2024/11/22-18:34:56.919379 140025196639808 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220746.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:34:58.007746 140025196639808 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220746.sst): second pass time = 2.37's, 45.633'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.09's, 34.607'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3202.421'MB/sec (index lex order gen) rebuild zvType time = 0.86's, 0.268'MB/sec write SST data time = 0.23's, 165.708'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33387.097'MB/sec zip my value time = 2.37's, unzip length = 0.108'GB zip my value throughput = 45.633'MB/sec zip pipeline throughput = 150.666'MB/sec entries = 923526 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.33 avg-zval = 34.38 usrkeys = 923526 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.33 avg-zval = 34.38 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923526 bytes = 3116912 } seq expand size = 6464682 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5141 value = 3.4131 dict = 1.00 all = 3.5592 } Zip/UnZip{ index = 0.0951 value = 0.2930 dict = 1.00 all = 0.2810 } ---------------------------- total value len = 0.114824 GB avg = 0.124 KB (by entry num) total key len = 0.025859 GB avg = 0.028 KB total ukey len = 0.018471 GB avg = 0.020 KB total ukey num = 0.000923526 Billion total entry num = 0.000923526 Billion write speed all = 0.004024684 MB/sec (with seq num) write speed all = 0.003813320 MB/sec (without seq num) 2024/11/22-18:34:58.048330 140025196639808 [compaction_job.cc:1948] [default] [JOB 7647] Generated table #220746: 923526 keys, 37710402 bytes, temperature: kWarm 2024/11/22-18:34:58.048496 140025196639808 EVENT_LOG_v1 {"time_micros": 1732271698048378, "cf_name": "default", "job": 7647, "event": "table_file_creation", "file_number": 220746, "file_size": 37710402, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774735, "largest_seqno": 3468885225, "table_properties": {"data_size": 31748176, "index_size": 1756736, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25858728, "raw_average_key_size": 28, "raw_value_size": 108359068, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923526, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732271693, "oldest_key_time": 0, "file_creation_time": 1732271693, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 220746, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:34:58.118968 140025196639808 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220747.sst) /dev/shm/ToplingTemp/Topling-241942-16CivG 2024/11/22-18:34:58.961592 140025196639808 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220747.sst): first pass time = 0.84's, 159.294'MB/sec 2024/11/22-18:35:01.727144 140025196639808 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220747.sst): old prealloc_size = 147639500, real_size = 33844400 2024/11/22-18:35:01.727892 140025196639808 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220747.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926175 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0319 GB avg-val = 116.92 avg-zval = 34.39 2024/11/22-18:35:01.728631 140025196639808 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220747.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:35:02.846315 140025196639808 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220747.sst): second pass time = 2.76's, 39.164'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.12's, 33.798'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2416.855'MB/sec (index lex order gen) rebuild zvType time = 0.96's, 0.242'MB/sec write SST data time = 0.16's, 235.704'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 22690.256'MB/sec zip my value time = 2.76's, unzip length = 0.108'GB zip my value throughput = 39.164'MB/sec zip pipeline throughput = 96.762'MB/sec entries = 926175 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.92 avg-zval = 34.39 usrkeys = 926175 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.92 avg-zval = 34.39 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926175 bytes = 3125856 } seq expand size = 6483225 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.5129 value = 3.3998 dict = 1.00 all = 3.5484 } Zip/UnZip{ index = 0.0951 value = 0.2941 dict = 1.00 all = 0.2818 } ---------------------------- total value len = 0.229592 GB avg = 0.124 KB (by entry num) total key len = 0.051792 GB avg = 0.028 KB total ukey len = 0.036994 GB avg = 0.020 KB total ukey num = 0.001849701 Billion total entry num = 0.001849701 Billion write speed all = 0.008048786 MB/sec (with seq num) write speed all = 0.007625510 MB/sec (without seq num) 2024/11/22-18:35:02.880115 140025196639808 [compaction_job.cc:1948] [default] [JOB 7647] Generated table #220747: 926175 keys, 37825102 bytes, temperature: kWarm 2024/11/22-18:35:02.880187 140025196639808 EVENT_LOG_v1 {"time_micros": 1732271702880151, "cf_name": "default", "job": 7647, "event": "table_file_creation", "file_number": 220747, "file_size": 37825102, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774766, "largest_seqno": 3468885592, "table_properties": {"data_size": 31850720, "index_size": 1761984, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25932900, "raw_average_key_size": 28, "raw_value_size": 108284930, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926175, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732271698, "oldest_key_time": 0, "file_creation_time": 1732271698, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 220747, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:35:02.921743 140025196639808 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220748.sst) /dev/shm/ToplingTemp/Topling-241942-9Q7cRJ 2024/11/22-18:35:03.953861 140025196639808 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220748.sst): first pass time = 1.03's, 130.048'MB/sec 2024/11/22-18:35:06.140083 140025196639808 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220748.sst): old prealloc_size = 147639500, real_size = 33684480 2024/11/22-18:35:06.140622 140025196639808 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220748.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922842 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0317 GB avg-val = 117.44 avg-zval = 34.34 2024/11/22-18:35:06.141336 140025196639808 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220748.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:35:07.126954 140025196639808 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220748.sst): second pass time = 2.19's, 49.586'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.99's, 38.159'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3409.491'MB/sec (index lex order gen) rebuild zvType time = 0.85's, 0.272'MB/sec write SST data time = 0.14's, 274.676'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32290.680'MB/sec zip my value time = 2.19's, unzip length = 0.108'GB zip my value throughput = 49.586'MB/sec zip pipeline throughput = 93.602'MB/sec entries = 922842 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.44 avg-zval = 34.34 usrkeys = 922842 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.44 avg-zval = 34.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922842 bytes = 3114608 } seq expand size = 6459894 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.4470 value = 3.4203 dict = 1.00 all = 3.5642 } Zip/UnZip{ index = 0.0957 value = 0.2924 dict = 1.00 all = 0.2806 } ---------------------------- total value len = 0.344430 GB avg = 0.124 KB (by entry num) total key len = 0.077631 GB avg = 0.028 KB total ukey len = 0.055451 GB avg = 0.020 KB total ukey num = 0.002772543 Billion total entry num = 0.002772543 Billion write speed all = 0.012071296 MB/sec (with seq num) write speed all = 0.011436920 MB/sec (without seq num) 2024/11/22-18:35:07.161480 140025196639808 [compaction_job.cc:1948] [default] [JOB 7647] Generated table #220748: 922842 keys, 37657318 bytes, temperature: kWarm 2024/11/22-18:35:07.161547 140025196639808 EVENT_LOG_v1 {"time_micros": 1732271707161513, "cf_name": "default", "job": 7647, "event": "table_file_creation", "file_number": 220748, "file_size": 37657318, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774802, "largest_seqno": 3468885432, "table_properties": {"data_size": 31686896, "index_size": 1766720, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25839576, "raw_average_key_size": 28, "raw_value_size": 108378156, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922842, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732271702, "oldest_key_time": 0, "file_creation_time": 1732271702, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 220748, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:35:07.197666 140025196639808 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220749.sst) /dev/shm/ToplingTemp/Topling-241942-cH6r0K 2024/11/22-18:35:08.335134 140025196639808 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220749.sst): first pass time = 1.14's, 118.002'MB/sec 2024/11/22-18:35:10.529603 140025196639808 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220749.sst): old prealloc_size = 147639500, real_size = 34985248 2024/11/22-18:35:10.530218 140025196639808 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220749.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1083765 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1039 GB zip-val = 0.0326 GB avg-val = 95.84 avg-zval = 30.10 2024/11/22-18:35:10.530474 140025196639808 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220749.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:35:11.320313 140025196639808 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220749.sst): second pass time = 2.19's, 47.352'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.79's, 46.003'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3530.644'MB/sec (index lex order gen) rebuild zvType time = 0.53's, 0.508'MB/sec write SST data time = 0.26's, 141.777'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.032'MB, throughput = 30868.419'MB/sec zip my value time = 2.19's, unzip length = 0.104'GB zip my value throughput = 47.352'MB/sec zip pipeline throughput = 91.034'MB/sec entries = 1083765 avg-key = 28.00 avg-zkey = 1.93 avg-val = 95.84 avg-zval = 30.10 usrkeys = 1083765 avg-key = 20.00 avg-zkey = 1.93 avg-val = 96.77 avg-zval = 30.10 TagRS{ kinds = RS_Key0_TagN bytes = 169440 } TagArray{ size = 142853 bytes = 446432 } seq expand size = 999971 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0217 GB value = 0.1039 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0326 GB dict = 1.03 MB all = 0.0364 GB } UnZip/Zip{ index = 10.3796 value = 3.1837 dict = 1.00 all = 3.6900 } Zip/UnZip{ index = 0.0963 value = 0.3141 dict = 1.00 all = 0.2710 } ---------------------------- total value len = 0.449302 GB avg = 0.117 KB (by entry num) total key len = 0.107977 GB avg = 0.028 KB total ukey len = 0.077126 GB avg = 0.020 KB total ukey num = 0.003856308 Billion total entry num = 0.003856308 Billion write speed all = 0.015936722 MB/sec (with seq num) write speed all = 0.015054480 MB/sec (without seq num) 2024/11/22-18:35:11.355432 140025196639808 [compaction_job.cc:1948] [default] [JOB 7647] Generated table #220749: 1083765 keys, 36373256 bytes, temperature: kWarm 2024/11/22-18:35:11.355509 140025196639808 EVENT_LOG_v1 {"time_micros": 1732271711355468, "cf_name": "default", "job": 7647, "event": "table_file_creation", "file_number": 220749, "file_size": 36373256, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3468885174, "table_properties": {"data_size": 32625872, "index_size": 2088256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30345420, "raw_average_key_size": 28, "raw_value_size": 103872335, "raw_average_value_size": 95, "num_data_blocks": 1, "num_entries": 1083765, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732271707, "oldest_key_time": 0, "file_creation_time": 1732271707, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 220749, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:35:11.432017 140025196639808 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220750.sst) /dev/shm/ToplingTemp/Topling-241942-CKe5GI 2024/11/22-18:35:12.723817 140025196639808 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220750.sst): first pass time = 1.29's, 103.904'MB/sec 2024/11/22-18:35:14.588342 140025196639808 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220750.sst): old prealloc_size = 147639500, real_size = 35518288 2024/11/22-18:35:14.588892 140025196639808 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220750.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923658 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0335 GB avg-val = 117.31 avg-zval = 36.30 2024/11/22-18:35:14.589333 140025196639808 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220750.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:35:15.394273 140025196639808 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220750.sst): second pass time = 1.86's, 58.136'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 47.388'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3379.025'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.341'MB/sec write SST data time = 0.13's, 298.111'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34036.910'MB/sec zip my value time = 1.86's, unzip length = 0.108'GB zip my value throughput = 58.136'MB/sec zip pipeline throughput = 60.485'MB/sec entries = 923658 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.31 avg-zval = 36.30 usrkeys = 923658 avg-key = 20.00 avg-zkey = 1.90 avg-val = 120.92 avg-zval = 36.30 TagRS{ kinds = RS_Key0_TagN bytes = 144440 } TagArray{ size = 476167 bytes = 1666592 } seq expand size = 3333169 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0335 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.5129 value = 3.2316 dict = 1.00 all = 3.5144 } Zip/UnZip{ index = 0.0951 value = 0.3094 dict = 1.00 all = 0.2845 } ---------------------------- total value len = 0.560991 GB avg = 0.117 KB (by entry num) total key len = 0.133839 GB avg = 0.028 KB total ukey len = 0.095599 GB avg = 0.020 KB total ukey num = 0.004779966 Billion total entry num = 0.004779966 Billion write speed all = 0.019868008 MB/sec (with seq num) write speed all = 0.018774579 MB/sec (without seq num) 2024/11/22-18:35:15.429269 140025196639808 [compaction_job.cc:1948] [default] [JOB 7647] Generated table #220750: 923658 keys, 38190595 bytes, temperature: kWarm 2024/11/22-18:35:15.429330 140025196639808 EVENT_LOG_v1 {"time_micros": 1732271715429298, "cf_name": "default", "job": 7647, "event": "table_file_creation", "file_number": 220750, "file_size": 38190595, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3468885659, "table_properties": {"data_size": 33529984, "index_size": 1757184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25862424, "raw_average_key_size": 28, "raw_value_size": 108355428, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923658, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732271711, "oldest_key_time": 0, "file_creation_time": 1732271711, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 220750, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:35:15.463932 140025196639808 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220751.sst) /dev/shm/ToplingTemp/Topling-241942-VFoUHH 2024/11/22-18:35:17.026226 140025196639808 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220751.sst): first pass time = 1.56's, 85.913'MB/sec 2024/11/22-18:35:19.020792 140025196639808 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220751.sst): old prealloc_size = 147639500, real_size = 34199616 2024/11/22-18:35:19.021326 140025196639808 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220751.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925551 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0322 GB avg-val = 117.01 avg-zval = 34.80 2024/11/22-18:35:19.022059 140025196639808 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220751.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:35:20.117179 140025196639808 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220751.sst): second pass time = 1.99's, 54.317'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.10's, 34.821'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3412.673'MB/sec (index lex order gen) rebuild zvType time = 0.86's, 0.269'MB/sec write SST data time = 0.23's, 162.616'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34435.711'MB/sec zip my value time = 1.99's, unzip length = 0.108'GB zip my value throughput = 54.317'MB/sec zip pipeline throughput = 130.168'MB/sec entries = 925551 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.01 avg-zval = 34.80 usrkeys = 925551 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.01 avg-zval = 34.80 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925551 bytes = 3123744 } seq expand size = 6478857 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0322 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.5119 value = 3.3627 dict = 1.00 all = 3.5156 } Zip/UnZip{ index = 0.0951 value = 0.2974 dict = 1.00 all = 0.2844 } ---------------------------- total value len = 0.675772 GB avg = 0.118 KB (by entry num) total key len = 0.159754 GB avg = 0.028 KB total ukey len = 0.114110 GB avg = 0.020 KB total ukey num = 0.005705517 Billion total entry num = 0.005705517 Billion write speed all = 0.023887872 MB/sec (with seq num) write speed all = 0.022582897 MB/sec (without seq num) 2024/11/22-18:35:20.156415 140025196639808 [compaction_job.cc:1948] [default] [JOB 7647] Generated table #220751: 925551 keys, 38178194 bytes, temperature: kWarm 2024/11/22-18:35:20.156494 140025196639808 EVENT_LOG_v1 {"time_micros": 1732271720156451, "cf_name": "default", "job": 7647, "event": "table_file_creation", "file_number": 220751, "file_size": 38178194, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3365258144, "largest_seqno": 3468885671, "table_properties": {"data_size": 32207088, "index_size": 1760960, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25915428, "raw_average_key_size": 28, "raw_value_size": 108302402, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925551, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732271715, "oldest_key_time": 0, "file_creation_time": 1732271715, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 220751, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:35:20.193699 140025196639808 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220752.sst) /dev/shm/ToplingTemp/Topling-241942-CupP9J 2024/11/22-18:35:21.083462 140025196639808 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220752.sst): first pass time = 0.89's, 150.854'MB/sec 2024/11/22-18:35:22.599210 140025196639808 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220752.sst): old prealloc_size = 147639500, real_size = 33904592 2024/11/22-18:35:22.599719 140025196639808 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220752.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922643 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0319 GB avg-val = 117.47 avg-zval = 34.59 2024/11/22-18:35:22.600483 140025196639808 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220752.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:35:23.672896 140025196639808 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220752.sst): second pass time = 1.51's, 71.565'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.07's, 35.385'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3677.038'MB/sec (index lex order gen) rebuild zvType time = 0.94's, 0.246'MB/sec write SST data time = 0.14's, 279.908'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34335.383'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.565'MB/sec zip pipeline throughput = 71.565'MB/sec entries = 922643 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.47 avg-zval = 34.59 usrkeys = 922643 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.47 avg-zval = 34.59 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922643 bytes = 3229264 } seq expand size = 6458501 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.5121 value = 3.3957 dict = 1.00 all = 3.5328 } Zip/UnZip{ index = 0.0951 value = 0.2945 dict = 1.00 all = 0.2831 } ---------------------------- total value len = 0.790614 GB avg = 0.119 KB (by entry num) total key len = 0.185588 GB avg = 0.028 KB total ukey len = 0.132563 GB avg = 0.020 KB total ukey num = 0.006628160 Billion total entry num = 0.006628160 Billion write speed all = 0.027906999 MB/sec (with seq num) write speed all = 0.026391149 MB/sec (without seq num) 2024/11/22-18:35:23.706671 140025196639808 [compaction_job.cc:1948] [default] [JOB 7647] Generated table #220752: 922643 keys, 37992152 bytes, temperature: kWarm 2024/11/22-18:35:23.706731 140025196639808 EVENT_LOG_v1 {"time_micros": 1732271723706699, "cf_name": "default", "job": 7647, "event": "table_file_creation", "file_number": 220752, "file_size": 37992152, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3468885690, "table_properties": {"data_size": 31918336, "index_size": 1755392, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25834004, "raw_average_key_size": 28, "raw_value_size": 108383810, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922643, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732271720, "oldest_key_time": 0, "file_creation_time": 1732271720, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 220752, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:35:23.742395 140025196639808 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220753.sst) /dev/shm/ToplingTemp/Topling-241942-cng2CK 2024/11/22-18:35:24.737062 140025196639808 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220753.sst): first pass time = 0.99's, 134.942'MB/sec 2024/11/22-18:35:26.349476 140025196639808 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220753.sst): old prealloc_size = 147639500, real_size = 33732320 2024/11/22-18:35:26.350043 140025196639808 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220753.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922783 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0317 GB avg-val = 117.45 avg-zval = 34.39 2024/11/22-18:35:26.350801 140025196639808 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220753.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:35:27.275875 140025196639808 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220753.sst): second pass time = 1.61's, 67.242'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.826'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3248.337'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.289'MB/sec write SST data time = 0.13's, 295.964'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 27264.658'MB/sec zip my value time = 1.61's, unzip length = 0.108'GB zip my value throughput = 67.242'MB/sec zip pipeline throughput = 109.407'MB/sec entries = 922783 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.45 avg-zval = 34.39 usrkeys = 922783 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.45 avg-zval = 34.39 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922783 bytes = 3229760 } seq expand size = 6459481 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.4433 value = 3.4152 dict = 1.00 all = 3.5488 } Zip/UnZip{ index = 0.0958 value = 0.2928 dict = 1.00 all = 0.2818 } ---------------------------- total value len = 0.905454 GB avg = 0.120 KB (by entry num) total key len = 0.211426 GB avg = 0.028 KB total ukey len = 0.151019 GB avg = 0.020 KB total ukey num = 0.007550943 Billion total entry num = 0.007550943 Billion write speed all = 0.031925292 MB/sec (with seq num) write speed all = 0.030198582 MB/sec (without seq num) 2024/11/22-18:35:27.309847 140025196639808 [compaction_job.cc:1948] [default] [JOB 7647] Generated table #220753: 922783 keys, 37820266 bytes, temperature: kWarm 2024/11/22-18:35:27.309906 140025196639808 EVENT_LOG_v1 {"time_micros": 1732271727309876, "cf_name": "default", "job": 7647, "event": "table_file_creation", "file_number": 220753, "file_size": 37820266, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701122, "largest_seqno": 3468885705, "table_properties": {"data_size": 31734224, "index_size": 1767232, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25837924, "raw_average_key_size": 28, "raw_value_size": 108379854, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922783, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732271723, "oldest_key_time": 0, "file_creation_time": 1732271723, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 220753, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:35:27.364268 140025196639808 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220754.sst) /dev/shm/ToplingTemp/Topling-241942-ZLlrCK 2024/11/22-18:35:28.285122 140025196639808 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220754.sst): first pass time = 0.92's, 145.760'MB/sec 2024/11/22-18:35:29.611872 140025196639808 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220754.sst): old prealloc_size = 147639500, real_size = 33443792 2024/11/22-18:35:29.612409 140025196639808 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220754.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924039 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 117.25 avg-zval = 34.04 2024/11/22-18:35:29.613162 140025196639808 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220754.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:35:30.547305 140025196639808 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220754.sst): second pass time = 1.32's, 81.770'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 40.003'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3410.721'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.286'MB/sec write SST data time = 0.13's, 291.173'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 35926.902'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.770'MB/sec zip pipeline throughput = 81.770'MB/sec entries = 924039 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.25 avg-zval = 34.04 usrkeys = 924039 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.25 avg-zval = 34.04 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924039 bytes = 3118640 } seq expand size = 6468273 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5119 value = 3.4445 dict = 1.00 all = 3.5868 } Zip/UnZip{ index = 0.0951 value = 0.2903 dict = 1.00 all = 0.2788 } ---------------------------- total value len = 1.020267 GB avg = 0.120 KB (by entry num) total key len = 0.237299 GB avg = 0.028 KB total ukey len = 0.169500 GB avg = 0.020 KB total ukey num = 0.008474982 Billion total entry num = 0.008474982 Billion write speed all = 0.035943350 MB/sec (with seq num) write speed all = 0.034005516 MB/sec (without seq num) 2024/11/22-18:35:30.583739 140025196639808 [compaction_job.cc:1948] [default] [JOB 7647] Generated table #220754: 924039 keys, 37419494 bytes, temperature: kWarm 2024/11/22-18:35:30.583809 140025196639808 EVENT_LOG_v1 {"time_micros": 1732271730583771, "cf_name": "default", "job": 7647, "event": "table_file_creation", "file_number": 220754, "file_size": 37419494, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369194706, "largest_seqno": 3468885738, "table_properties": {"data_size": 31454528, "index_size": 1758080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25873092, "raw_average_key_size": 28, "raw_value_size": 108344670, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924039, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732271727, "oldest_key_time": 0, "file_creation_time": 1732271727, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 220754, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:35:30.621161 140025196639808 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220755.sst) /dev/shm/ToplingTemp/Topling-241942-KptbZJ 2024/11/22-18:35:31.444671 140025196639808 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220755.sst): first pass time = 0.82's, 162.991'MB/sec 2024/11/22-18:35:32.763468 140025196639808 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220755.sst): old prealloc_size = 147639500, real_size = 31768480 2024/11/22-18:35:32.764012 140025196639808 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220755.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924262 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0298 GB avg-val = 117.22 avg-zval = 32.22 2024/11/22-18:35:32.766466 140025196639808 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220755.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:35:33.635952 140025196639808 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220755.sst): second pass time = 1.32's, 82.190'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 41.102'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3362.393'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.311'MB/sec write SST data time = 0.13's, 279.248'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 32034.942'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.190'MB/sec zip pipeline throughput = 82.190'MB/sec entries = 924262 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.22 avg-zval = 32.22 usrkeys = 924262 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.22 avg-zval = 32.22 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924262 bytes = 3234928 } seq expand size = 6469834 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0298 GB dict = 1.08 MB all = 0.0359 GB } UnZip/Zip{ index = 10.5114 value = 3.6381 dict = 1.00 all = 3.7428 } Zip/UnZip{ index = 0.0951 value = 0.2749 dict = 1.00 all = 0.2672 } ---------------------------- total value len = 1.135075 GB avg = 0.121 KB (by entry num) total key len = 0.263179 GB avg = 0.028 KB total ukey len = 0.187985 GB avg = 0.020 KB total ukey num = 0.009399244 Billion total entry num = 0.009399244 Billion write speed all = 0.039960909 MB/sec (with seq num) write speed all = 0.037811930 MB/sec (without seq num) 2024/11/22-18:35:33.668425 140025196639808 [compaction_job.cc:1948] [default] [JOB 7647] Generated table #220755: 924262 keys, 35859948 bytes, temperature: kWarm 2024/11/22-18:35:33.668507 140025196639808 EVENT_LOG_v1 {"time_micros": 1732271733668457, "cf_name": "default", "job": 7647, "event": "table_file_creation", "file_number": 220755, "file_size": 35859948, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3326330223, "largest_seqno": 3468885749, "table_properties": {"data_size": 29778640, "index_size": 1758592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25879336, "raw_average_key_size": 28, "raw_value_size": 108338424, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924262, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732271730, "oldest_key_time": 0, "file_creation_time": 1732271730, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 220755, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:35:33.704771 140025196639808 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220756.sst) /dev/shm/ToplingTemp/Topling-241942-NTg9oK 2024/11/22-18:35:34.570043 140025196639808 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220756.sst): first pass time = 0.87's, 155.124'MB/sec 2024/11/22-18:35:35.889528 140025196639808 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220756.sst): old prealloc_size = 147639500, real_size = 33759312 2024/11/22-18:35:35.890085 140025196639808 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220756.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923168 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0318 GB avg-val = 117.39 avg-zval = 34.40 2024/11/22-18:35:35.892531 140025196639808 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220756.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:35:36.809711 140025196639808 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220756.sst): second pass time = 1.32's, 82.198'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 41.132'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3298.209'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.292'MB/sec write SST data time = 0.13's, 293.469'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32712.706'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.198'MB/sec zip pipeline throughput = 82.198'MB/sec entries = 923168 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.39 avg-zval = 34.40 usrkeys = 923168 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.39 avg-zval = 34.40 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923168 bytes = 3231104 } seq expand size = 6462176 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.4453 value = 3.4120 dict = 1.00 all = 3.5462 } Zip/UnZip{ index = 0.0957 value = 0.2931 dict = 1.00 all = 0.2820 } ---------------------------- total value len = 1.249906 GB avg = 0.121 KB (by entry num) total key len = 0.289028 GB avg = 0.028 KB total ukey len = 0.206448 GB avg = 0.020 KB total ukey num = 0.010322412 Billion total entry num = 0.010322412 Billion write speed all = 0.043977434 MB/sec (with seq num) write speed all = 0.041617601 MB/sec (without seq num) 2024/11/22-18:35:36.842378 140025196639808 [compaction_job.cc:1948] [default] [JOB 7647] Generated table #220756: 923168 keys, 37848312 bytes, temperature: kWarm 2024/11/22-18:35:36.842439 140025196639808 EVENT_LOG_v1 {"time_micros": 1732271736842408, "cf_name": "default", "job": 7647, "event": "table_file_creation", "file_number": 220756, "file_size": 37848312, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701165, "largest_seqno": 3468885760, "table_properties": {"data_size": 31760704, "index_size": 1767616, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25848704, "raw_average_key_size": 28, "raw_value_size": 108369048, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923168, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732271733, "oldest_key_time": 0, "file_creation_time": 1732271733, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 220756, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:35:36.880299 140025196639808 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220757.sst) /dev/shm/ToplingTemp/Topling-241942-NcOGcJ 2024/11/22-18:35:37.666556 140025196639808 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220757.sst): first pass time = 0.79's, 170.713'MB/sec 2024/11/22-18:35:38.985689 140025196639808 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220757.sst): old prealloc_size = 147639500, real_size = 33411968 2024/11/22-18:35:38.986234 140025196639808 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220757.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927027 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.78 avg-zval = 33.90 2024/11/22-18:35:38.987734 140025196639808 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220757.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:35:39.899059 140025196639808 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220757.sst): second pass time = 1.32's, 82.107'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 40.820'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3350.049'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.297'MB/sec write SST data time = 0.13's, 282.251'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33261.832'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.107'MB/sec zip pipeline throughput = 82.107'MB/sec entries = 927027 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.78 avg-zval = 33.90 usrkeys = 927027 avg-key = 20.00 avg-zkey = 1.89 avg-val = 123.20 avg-zval = 33.90 TagRS{ kinds = RS_Key0_TagN bytes = 144960 } TagArray{ size = 849881 bytes = 2868368 } seq expand size = 5949167 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5806 value = 3.4448 dict = 1.00 all = 3.5999 } Zip/UnZip{ index = 0.0945 value = 0.2903 dict = 1.00 all = 0.2778 } ---------------------------- total value len = 1.364116 GB avg = 0.121 KB (by entry num) total key len = 0.314984 GB avg = 0.028 KB total ukey len = 0.224989 GB avg = 0.020 KB total ukey num = 0.011249439 Billion total entry num = 0.011249439 Billion write speed all = 0.047978690 MB/sec (with seq num) write speed all = 0.045407155 MB/sec (without seq num) 2024/11/22-18:35:39.935849 140025196639808 [compaction_job.cc:1948] [default] [JOB 7647] Generated table #220757: 927027 keys, 37283266 bytes, temperature: kWarm 2024/11/22-18:35:39.935909 140025196639808 EVENT_LOG_v1 {"time_micros": 1732271739935879, "cf_name": "default", "job": 7647, "event": "table_file_creation", "file_number": 220757, "file_size": 37283266, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3468885790, "table_properties": {"data_size": 31427696, "index_size": 1752320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25956756, "raw_average_key_size": 28, "raw_value_size": 108261114, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927027, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732271736, "oldest_key_time": 0, "file_creation_time": 1732271736, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 220757, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:35:39.973734 140025196639808 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220758.sst) /dev/shm/ToplingTemp/Topling-241942-7tdnIK 2024/11/22-18:35:40.723395 140025196639808 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220758.sst): first pass time = 0.75's, 179.047'MB/sec 2024/11/22-18:35:42.147979 140025196639808 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220758.sst): old prealloc_size = 147639500, real_size = 36430688 2024/11/22-18:35:42.148510 140025196639808 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220758.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924356 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0344 GB avg-val = 117.20 avg-zval = 37.26 2024/11/22-18:35:42.148837 140025196639808 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220758.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:35:42.877611 140025196639808 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220758.sst): second pass time = 1.42's, 76.081'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.73's, 52.676'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3467.478'MB/sec (index lex order gen) rebuild zvType time = 0.60's, 0.384'MB/sec write SST data time = 0.13's, 300.805'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 30358.344'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 76.081'MB/sec zip pipeline throughput = 76.081'MB/sec entries = 924356 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.20 avg-zval = 37.26 usrkeys = 924356 avg-key = 20.00 avg-zkey = 1.90 avg-val = 119.53 avg-zval = 37.26 TagRS{ kinds = RS_Key0_TagN bytes = 144520 } TagArray{ size = 307203 bytes = 998432 } seq expand size = 2150421 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.5128 value = 3.1456 dict = 1.00 all = 3.4922 } Zip/UnZip{ index = 0.0951 value = 0.3179 dict = 1.00 all = 0.2864 } ---------------------------- total value len = 1.474603 GB avg = 0.121 KB (by entry num) total key len = 0.340866 GB avg = 0.028 KB total ukey len = 0.243476 GB avg = 0.020 KB total ukey num = 0.012173795 Billion total entry num = 0.012173795 Billion write speed all = 0.051870868 MB/sec (with seq num) write speed all = 0.049088269 MB/sec (without seq num) 2024/11/22-18:35:42.913972 140025196639808 [compaction_job.cc:1948] [default] [JOB 7647] Generated table #220758: 924356 keys, 38433659 bytes, temperature: kWarm 2024/11/22-18:35:42.914034 140025196639808 EVENT_LOG_v1 {"time_micros": 1732271742914002, "cf_name": "default", "job": 7647, "event": "table_file_creation", "file_number": 220758, "file_size": 38433659, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3332052738, "table_properties": {"data_size": 34440912, "index_size": 1758528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25881968, "raw_average_key_size": 28, "raw_value_size": 108335900, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924356, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732271739, "oldest_key_time": 0, "file_creation_time": 1732271739, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 220758, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:35:42.949561 140025196639808 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220759.sst) /dev/shm/ToplingTemp/Topling-241942-KQDUEK 2024/11/22-18:35:43.812509 140025196639808 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220759.sst): first pass time = 0.86's, 155.541'MB/sec 2024/11/22-18:35:45.133682 140025196639808 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220759.sst): old prealloc_size = 147639500, real_size = 33439600 2024/11/22-18:35:45.134270 140025196639808 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220759.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922629 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0314 GB avg-val = 117.47 avg-zval = 34.08 2024/11/22-18:35:45.136351 140025196639808 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220759.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:35:46.055393 140025196639808 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220759.sst): second pass time = 1.32's, 82.079'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.715'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3098.744'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.291'MB/sec write SST data time = 0.13's, 290.834'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 35781.287'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.079'MB/sec zip pipeline throughput = 82.079'MB/sec entries = 922629 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.47 avg-zval = 34.08 usrkeys = 922629 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.47 avg-zval = 34.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922629 bytes = 3229216 } seq expand size = 6458403 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4461 value = 3.4471 dict = 1.00 all = 3.5766 } Zip/UnZip{ index = 0.0957 value = 0.2901 dict = 1.00 all = 0.2796 } ---------------------------- total value len = 1.589445 GB avg = 0.121 KB (by entry num) total key len = 0.366700 GB avg = 0.028 KB total ukey len = 0.261928 GB avg = 0.020 KB total ukey num = 0.013096424 Billion total entry num = 0.013096424 Billion write speed all = 0.055885140 MB/sec (with seq num) write speed all = 0.052891925 MB/sec (without seq num) 2024/11/22-18:35:46.088113 140025196639808 [compaction_job.cc:1948] [default] [JOB 7647] Generated table #220759: 922629 keys, 37527012 bytes, temperature: kWarm 2024/11/22-18:35:46.088175 140025196639808 EVENT_LOG_v1 {"time_micros": 1732271746088144, "cf_name": "default", "job": 7647, "event": "table_file_creation", "file_number": 220759, "file_size": 37527012, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3332052739, "largest_seqno": 3467891288, "table_properties": {"data_size": 31442272, "index_size": 1766464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25833612, "raw_average_key_size": 28, "raw_value_size": 108384222, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922629, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732271742, "oldest_key_time": 0, "file_creation_time": 1732271742, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 220759, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:35:46.124746 140025196639808 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220760.sst) /dev/shm/ToplingTemp/Topling-241942-XE1hTK 2024/11/22-18:35:46.316558 140025196639808 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220760.sst): first pass time = 0.19's, 249.385'MB/sec 2024/11/22-18:35:46.789037 140025196639808 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220760.sst): old prealloc_size = 147639500, real_size = 13360288 2024/11/22-18:35:46.789261 140025196639808 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220760.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 330826 prefix = 4 raw-key = 0.0053 GB zip-key = 0.0006 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0386 GB zip-val = 0.0126 GB avg-val = 116.57 avg-zval = 38.23 2024/11/22-18:35:46.789557 140025196639808 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220760.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0008 GB, waited 0.000 sec, Key+Value = 0.048 GB 2024/11/22-18:35:47.120860 140025196639808 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00/220760.sst): second pass time = 0.47's, 81.716'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.33's, 44.705'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3151.332'MB/sec (index lex order gen) rebuild zvType time = 0.27's, 0.309'MB/sec write SST data time = 0.06's, 232.523'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.395'MB, throughput = 15070.952'MB/sec zip my value time = 0.47's, unzip length = 0.039'GB zip my value throughput = 81.716'MB/sec zip pipeline throughput = 81.716'MB/sec entries = 330826 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.57 avg-zval = 38.23 usrkeys = 330826 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.57 avg-zval = 38.23 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 330826 bytes = 1157904 } seq expand size = 2315782 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0066 GB value = 0.0386 GB dict = 0.40 MB all = 0.0478 GB } __ZipSize{ index = 0.0006 GB value = 0.0126 GB dict = 0.40 MB all = 0.0148 GB } UnZip/Zip{ index = 10.4936 value = 3.0492 dict = 1.00 all = 3.2242 } Zip/UnZip{ index = 0.0953 value = 0.3280 dict = 1.00 all = 0.3102 } ---------------------------- total value len = 1.630324 GB avg = 0.121 KB (by entry num) total key len = 0.375963 GB avg = 0.028 KB total ukey len = 0.268545 GB avg = 0.020 KB total ukey num = 0.013427250 Billion total entry num = 0.013427250 Billion write speed all = 0.057315908 MB/sec (with seq num) write speed all = 0.054247176 MB/sec (without seq num) 2024/11/22-18:35:47.138847 140025196639808 [compaction_job.cc:1948] [default] [JOB 7647] Generated table #220760: 330826 keys, 14833583 bytes, temperature: kWarm 2024/11/22-18:35:47.138913 140025196639808 EVENT_LOG_v1 {"time_micros": 1732271747138883, "cf_name": "default", "job": 7647, "event": "table_file_creation", "file_number": 220760, "file_size": 14833583, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701530, "largest_seqno": 3467372993, "table_properties": {"data_size": 12646864, "index_size": 630528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 9263128, "raw_average_key_size": 28, "raw_value_size": 38563284, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 330826, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732271746, "oldest_key_time": 0, "file_creation_time": 1732271746, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 220760, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:35:47.152858 140025196639808 [compaction_job.cc:703] job-07647: subcompact[0], size: 0.540953 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07647/att-00 [220746,220747,220748,220749,220750,220751,220752,220753,220754,220755,220756,220757,220758,220759,220760] 2024/11/22-18:35:47.248867 140025196639808 (Original Log Time 2024/11/22-18:34:52.956306) EVENT_LOG_v1 {"time_micros": 1732271692956262, "job": 7647, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [220734, 220739], "files_L3": [220703, 220704, 220705, 220706, 220707, 220708, 220709, 220710, 220711, 220681, 220682, 220683, 220684, 220685, 220686], "score": 0.953911, "input_data_size": 567355965, "oldest_snapshot_seqno": 3468914679} 2024/11/22-18:35:47.248882 140025196639808 (Original Log Time 2024/11/22-18:35:47.247644) [compaction_job.cc:2022] [default] [JOB 7647] Compacted 2@2 + 15@3 files to L3 => 540952559 bytes 2024/11/22-18:35:47.248885 140025196639808 (Original Log Time 2024/11/22-18:35:47.248348) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 10.5 rd, 10.0 wr, level 3, files in(2, 15) out(15 +0 blob) MB in(28.7, 512.4 +0.0 blob) out(515.9 +0.0 blob), read-write-amplify(36.9) write-amplify(18.0) OK, records in: 13544206, records dropped: 116956 output_compression: Snappy 2024/11/22-18:35:47.248891 140025196639808 (Original Log Time 2024/11/22-18:35:47.248845) EVENT_LOG_v1 {"time_micros": 1732271747248371, "cf": "default", "job": 7647, "event": "compaction_finished", "compaction_time_micros": 54196560, "compaction_time_cpu_micros": 23040574, "output_level": 3, "num_output_files": 15, "total_output_size": 540952559, "num_input_records": 13544206, "num_output_records": 13427250, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 366576915, "file_range_sync_nanos": 0, "file_fsync_nanos": 15284, "file_prepare_write_nanos": 39244, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/22-18:35:47.378032 140025196639808 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732271692,"executesMs":54433,"compactionJobId":7647,"attempt":0,"compactionInputRawBytes":1943483397,"compactionInputZipBytes":567355965,"compactionOutputRawBytes":0,"compactionOutputZipBytes":540952559,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-44cba5cf5b66ce3103195efc3f63ad45-8d3177d984fcc3fe-00"}}, response = 2024/11/22-18:35:47.378101 140025196639808 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-07647/att-00: olev 3, work 54.434 s, result 6.364 ms, install 1.269 ms, input{raw 1.810 GiB zip 541.073 MiB}