2024/11/24-05:54:37.447624 139957349447232 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00: fork to child time = 0.008023 sec 2024/11/24-05:54:37.448430 139957349447232 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00 2024/11/24-05:54:37.453567 139957349447232 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00 2024/11/24-05:54:37.454444 139957349447232 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-05:54:37.454644 139957349447232 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-05:54:37.506692 139957349447232 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-42486/att-00/MANIFEST-206347 2024/11/24-05:54:37.529852 139957349447232 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-42486/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 294184, last_sequence is 4468407194, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 293981 2024/11/24-05:54:37.529859 139957349447232 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-05:54:37.529959 139957349447232 [version_set.cc:5721] Creating manifest 294184 2024/11/24-05:54:37.562258 139957349447232 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-05:54:37.583368 139957349447232 [compaction_job.cc:2437] [default] [JOB 42486] Compacting 1@2 + 22@3 files to L3, score 0.92, subcompactions 1 : 1 2024/11/24-05:54:37.583382 139957349447232 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 31849 Base level 2, inputs: [294120(27MB)], [294157(5008KB) 294087(33MB) 294088(35MB) 294089(33MB) 294090(35MB) 294091(34MB) 294092(33MB) 294093(35MB) 294094(33MB) 294095(6706KB) 294044(33MB) 294045(35MB) 294046(33MB) 294047(35MB) 294048(33MB) 294049(34MB) 294050(35MB) 294051(33MB) 294052(35MB) 294053(33MB) 294054(33MB) 294158(35MB)] 2024/11/24-05:54:37.589829 139957349447232 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-05:54:37.589851 139957349447232 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-05:54:37.589866 139957349447232 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-05:54:37.589870 139957349447232 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-05:54:37.589880 139957349447232 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-05:54:37.903219 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294185.sst) /dev/shm/ToplingTemp/Topling-284055-cVOh34 2024/11/24-05:54:38.895643 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294185.sst): first pass time = 0.99's, 135.248'MB/sec 2024/11/24-05:54:40.281088 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294185.sst): old prealloc_size = 147639500, real_size = 31633440 2024/11/24-05:54:40.281653 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294185.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919581 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1085 GB zip-val = 0.0297 GB avg-val = 117.96 avg-zval = 32.25 2024/11/24-05:54:40.282440 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294185.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:54:41.100563 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294185.sst): second pass time = 1.38's, 78.386'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.581'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3195.294'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.13's, 283.966'MB/sec dict compress time = 0.00's, 1084920.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 36351.818'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.386'MB/sec zip pipeline throughput = 78.386'MB/sec entries = 919581 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.96 avg-zval = 32.25 usrkeys = 919581 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.96 avg-zval = 32.25 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919581 bytes = 3218544 } seq expand size = 6437067 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0297 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5144 value = 3.6578 dict = 1.00 all = 3.7583 } Zip/UnZip{ index = 0.0951 value = 0.2734 dict = 1.00 all = 0.2661 } ---------------------------- total value len = 2251.440480 GB avg = 0.200 KB (by entry num) total key len = 306.908711 GB avg = 0.027 KB total ukey len = 237.373119 GB avg = 0.015 KB total ukey num = 15.739580744 Billion total entry num = 11.280894692 Billion write speed all = 33.854530998 MB/sec (with seq num) write speed all = 32.660294026 MB/sec (without seq num) 2024/11/24-05:54:41.145865 139957349447232 [compaction_job.cc:1948] [default] [JOB 42486] Generated table #294185: 919581 keys, 35712712 bytes, temperature: kWarm 2024/11/24-05:54:41.145962 139957349447232 EVENT_LOG_v1 {"time_micros": 1732398881145902, "cf_name": "default", "job": 42486, "event": "table_file_creation", "file_number": 294185, "file_size": 35712712, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4268980586, "largest_seqno": 4467942533, "table_properties": {"data_size": 29654160, "index_size": 1749184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25748268, "raw_average_key_size": 28, "raw_value_size": 108469482, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 919581, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398877, "oldest_key_time": 0, "file_creation_time": 1732398877, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294185, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:54:41.185405 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294186.sst) /dev/shm/ToplingTemp/Topling-284055-kYRjk7 2024/11/24-05:54:42.207203 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294186.sst): first pass time = 1.02's, 131.360'MB/sec 2024/11/24-05:54:43.540795 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294186.sst): old prealloc_size = 147639500, real_size = 33155296 2024/11/24-05:54:43.541377 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294186.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927712 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.68 avg-zval = 33.59 2024/11/24-05:54:43.542133 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294186.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:54:44.393853 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294186.sst): second pass time = 1.33's, 81.215'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.676'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3156.185'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 295.801'MB/sec dict compress time = 0.00's, 1080340.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36300.528'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.215'MB/sec zip pipeline throughput = 81.215'MB/sec entries = 927712 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.68 avg-zval = 33.59 usrkeys = 927712 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.68 avg-zval = 33.59 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927712 bytes = 3247008 } seq expand size = 6493984 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5108 value = 3.4740 dict = 1.00 all = 3.6025 } Zip/UnZip{ index = 0.0951 value = 0.2879 dict = 1.00 all = 0.2776 } ---------------------------- total value len = 2251.555216 GB avg = 0.200 KB (by entry num) total key len = 306.934687 GB avg = 0.027 KB total ukey len = 237.391673 GB avg = 0.015 KB total ukey num = 15.740508456 Billion total entry num = 11.281822404 Billion write speed all = 33.854917636 MB/sec (with seq num) write speed all = 32.660634499 MB/sec (without seq num) 2024/11/24-05:54:44.439544 139957349447232 [compaction_job.cc:1948] [default] [JOB 42486] Generated table #294186: 927712 keys, 37256548 bytes, temperature: kWarm 2024/11/24-05:54:44.439624 139957349447232 EVENT_LOG_v1 {"time_micros": 1732398884439576, "cf_name": "default", "job": 42486, "event": "table_file_creation", "file_number": 294186, "file_size": 37256548, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178599, "largest_seqno": 4467942006, "table_properties": {"data_size": 31157968, "index_size": 1765248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25975936, "raw_average_key_size": 28, "raw_value_size": 108241872, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927712, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398881, "oldest_key_time": 0, "file_creation_time": 1732398881, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294186, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:54:44.476738 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294187.sst) /dev/shm/ToplingTemp/Topling-284055-Sac7P6 2024/11/24-05:54:45.465076 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294187.sst): first pass time = 0.99's, 135.806'MB/sec 2024/11/24-05:54:46.795855 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294187.sst): old prealloc_size = 147639500, real_size = 31412320 2024/11/24-05:54:46.796431 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294187.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 32.01 2024/11/24-05:54:46.797193 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294187.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:54:47.607876 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294187.sst): second pass time = 1.33's, 81.562'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.706'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3174.911'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 282.287'MB/sec dict compress time = 0.00's, 1085010.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 30918.132'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.562'MB/sec zip pipeline throughput = 81.562'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.01 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.01 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4474 value = 3.6869 dict = 1.00 all = 3.7820 } Zip/UnZip{ index = 0.0957 value = 0.2712 dict = 1.00 all = 0.2644 } ---------------------------- total value len = 2251.670128 GB avg = 0.200 KB (by entry num) total key len = 306.960427 GB avg = 0.027 KB total ukey len = 237.410059 GB avg = 0.015 KB total ukey num = 15.741427756 Billion total entry num = 11.282741704 Billion write speed all = 33.855338968 MB/sec (with seq num) write speed all = 32.661009309 MB/sec (without seq num) 2024/11/24-05:54:47.651236 139957349447232 [compaction_job.cc:1948] [default] [JOB 42486] Generated table #294187: 919300 keys, 35489010 bytes, temperature: kWarm 2024/11/24-05:54:47.651305 139957349447232 EVENT_LOG_v1 {"time_micros": 1732398887651270, "cf_name": "default", "job": 42486, "event": "table_file_creation", "file_number": 294187, "file_size": 35489010, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4315926014, "largest_seqno": 4466714167, "table_properties": {"data_size": 29422416, "index_size": 1759872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398884, "oldest_key_time": 0, "file_creation_time": 1732398884, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294187, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:54:47.693326 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294188.sst) /dev/shm/ToplingTemp/Topling-284055-V0nnf6 2024/11/24-05:54:48.571019 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294188.sst): first pass time = 0.88's, 152.927'MB/sec 2024/11/24-05:54:49.897887 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294188.sst): old prealloc_size = 147639500, real_size = 33264176 2024/11/24-05:54:49.898503 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294188.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927680 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.68 avg-zval = 33.68 2024/11/24-05:54:49.899273 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294188.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:54:50.746610 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294188.sst): second pass time = 1.33's, 81.629'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.032'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3009.085'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 297.245'MB/sec dict compress time = 0.00's, 1079846.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32596.172'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.629'MB/sec zip pipeline throughput = 81.629'MB/sec entries = 927680 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.68 avg-zval = 33.68 usrkeys = 927680 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.68 avg-zval = 33.68 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927680 bytes = 3246896 } seq expand size = 6493760 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.3769 value = 3.4644 dict = 1.00 all = 3.5916 } Zip/UnZip{ index = 0.0964 value = 0.2886 dict = 1.00 all = 0.2784 } ---------------------------- total value len = 2251.784865 GB avg = 0.200 KB (by entry num) total key len = 306.986402 GB avg = 0.027 KB total ukey len = 237.428613 GB avg = 0.015 KB total ukey num = 15.742355436 Billion total entry num = 11.283669384 Billion write speed all = 33.855794769 MB/sec (with seq num) write speed all = 32.661416514 MB/sec (without seq num) 2024/11/24-05:54:50.793059 139957349447232 [compaction_job.cc:1948] [default] [JOB 42486] Generated table #294188: 927680 keys, 37369526 bytes, temperature: kWarm 2024/11/24-05:54:50.793134 139957349447232 EVENT_LOG_v1 {"time_micros": 1732398890793096, "cf_name": "default", "job": 42486, "event": "table_file_creation", "file_number": 294188, "file_size": 37369526, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178645, "largest_seqno": 4467942388, "table_properties": {"data_size": 31244128, "index_size": 1787968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25975040, "raw_average_key_size": 28, "raw_value_size": 108242788, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927680, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398887, "oldest_key_time": 0, "file_creation_time": 1732398887, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294188, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:54:50.828992 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294189.sst) /dev/shm/ToplingTemp/Topling-284055-D7b2i5 2024/11/24-05:54:51.830836 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294189.sst): first pass time = 1.00's, 133.976'MB/sec 2024/11/24-05:54:53.139427 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294189.sst): old prealloc_size = 147639500, real_size = 30669056 2024/11/24-05:54:53.140026 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294189.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927335 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0287 GB avg-val = 116.73 avg-zval = 30.91 2024/11/24-05:54:53.140768 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294189.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:54:53.953363 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294189.sst): second pass time = 1.31's, 82.803'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 42.574'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3071.254'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.337'MB/sec write SST data time = 0.13's, 274.517'MB/sec dict compress time = 0.00's, 1080514.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 36878.870'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.803'MB/sec zip pipeline throughput = 82.803'MB/sec entries = 927335 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.73 avg-zval = 30.91 usrkeys = 927335 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.73 avg-zval = 30.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927335 bytes = 3129776 } seq expand size = 6491345 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0287 GB dict = 1.08 MB all = 0.0347 GB } UnZip/Zip{ index = 10.4445 value = 3.7770 dict = 1.00 all = 3.8734 } Zip/UnZip{ index = 0.0957 value = 0.2648 dict = 1.00 all = 0.2582 } ---------------------------- total value len = 2251.899608 GB avg = 0.200 KB (by entry num) total key len = 307.012368 GB avg = 0.027 KB total ukey len = 237.447159 GB avg = 0.015 KB total ukey num = 15.743282771 Billion total entry num = 11.284596719 Billion write speed all = 33.856220032 MB/sec (with seq num) write speed all = 32.661794298 MB/sec (without seq num) 2024/11/24-05:54:53.996436 139957349447232 [compaction_job.cc:1948] [default] [JOB 42486] Generated table #294189: 927335 keys, 34651474 bytes, temperature: kWarm 2024/11/24-05:54:53.996506 139957349447232 EVENT_LOG_v1 {"time_micros": 1732398893996470, "cf_name": "default", "job": 42486, "event": "table_file_creation", "file_number": 294189, "file_size": 34651474, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4364167302, "largest_seqno": 4467942812, "table_properties": {"data_size": 28661296, "index_size": 1775744, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25965380, "raw_average_key_size": 28, "raw_value_size": 108252398, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927335, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398890, "oldest_key_time": 0, "file_creation_time": 1732398890, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294189, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:54:54.034348 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294190.sst) /dev/shm/ToplingTemp/Topling-284055-kbik08 2024/11/24-05:54:55.013145 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294190.sst): first pass time = 0.98's, 137.130'MB/sec 2024/11/24-05:54:56.344499 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294190.sst): old prealloc_size = 147639500, real_size = 31515680 2024/11/24-05:54:56.345084 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294190.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919618 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 117.95 avg-zval = 32.11 2024/11/24-05:54:56.345846 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294190.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:54:57.155643 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294190.sst): second pass time = 1.33's, 81.547'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.884'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3147.714'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.12's, 286.485'MB/sec dict compress time = 0.00's, 1084990.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 26448.334'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.547'MB/sec zip pipeline throughput = 81.547'MB/sec entries = 919618 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.95 avg-zval = 32.11 usrkeys = 919618 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.95 avg-zval = 32.11 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919618 bytes = 3218672 } seq expand size = 6437326 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.08 MB all = 0.0356 GB } UnZip/Zip{ index = 10.4434 value = 3.6739 dict = 1.00 all = 3.7707 } Zip/UnZip{ index = 0.0958 value = 0.2722 dict = 1.00 all = 0.2652 } ---------------------------- total value len = 2252.014514 GB avg = 0.200 KB (by entry num) total key len = 307.038117 GB avg = 0.027 KB total ukey len = 237.465552 GB avg = 0.015 KB total ukey num = 15.744202389 Billion total entry num = 11.285516337 Billion write speed all = 33.856646542 MB/sec (with seq num) write speed all = 32.662174078 MB/sec (without seq num) 2024/11/24-05:54:57.198783 139957349447232 [compaction_job.cc:1948] [default] [JOB 42486] Generated table #294190: 919618 keys, 35595150 bytes, temperature: kWarm 2024/11/24-05:54:57.198854 139957349447232 EVENT_LOG_v1 {"time_micros": 1732398897198819, "cf_name": "default", "job": 42486, "event": "table_file_creation", "file_number": 294190, "file_size": 35595150, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178655, "largest_seqno": 4467941990, "table_properties": {"data_size": 29524432, "index_size": 1761152, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25749304, "raw_average_key_size": 28, "raw_value_size": 108468460, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 919618, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398894, "oldest_key_time": 0, "file_creation_time": 1732398894, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294190, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:54:57.235143 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294191.sst) /dev/shm/ToplingTemp/Topling-284055-unaYZ6 2024/11/24-05:54:58.249969 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294191.sst): first pass time = 1.01's, 132.261'MB/sec 2024/11/24-05:54:59.596261 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294191.sst): old prealloc_size = 147639500, real_size = 33137408 2024/11/24-05:54:59.596832 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294191.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927392 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 116.73 avg-zval = 33.58 2024/11/24-05:54:59.597615 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294191.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:55:00.445557 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294191.sst): second pass time = 1.35's, 80.455'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.847'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3227.989'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.12's, 299.573'MB/sec dict compress time = 0.00's, 1080478.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32504.377'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.455'MB/sec zip pipeline throughput = 80.455'MB/sec entries = 927392 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.73 avg-zval = 33.58 usrkeys = 927392 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.73 avg-zval = 33.58 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927392 bytes = 3245888 } seq expand size = 6491744 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5118 value = 3.4762 dict = 1.00 all = 3.6043 } Zip/UnZip{ index = 0.0951 value = 0.2877 dict = 1.00 all = 0.2774 } ---------------------------- total value len = 2252.129257 GB avg = 0.200 KB (by entry num) total key len = 307.064084 GB avg = 0.027 KB total ukey len = 237.484100 GB avg = 0.015 KB total ukey num = 15.745129781 Billion total entry num = 11.286443729 Billion write speed all = 33.857034488 MB/sec (with seq num) write speed all = 32.662515860 MB/sec (without seq num) 2024/11/24-05:55:00.492032 139957349447232 [compaction_job.cc:1948] [default] [JOB 42486] Generated table #294191: 927392 keys, 37237806 bytes, temperature: kWarm 2024/11/24-05:55:00.492105 139957349447232 EVENT_LOG_v1 {"time_micros": 1732398900492068, "cf_name": "default", "job": 42486, "event": "table_file_creation", "file_number": 294191, "file_size": 37237806, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178675, "largest_seqno": 4467942823, "table_properties": {"data_size": 31140912, "index_size": 1764480, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25966976, "raw_average_key_size": 28, "raw_value_size": 108250856, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927392, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398897, "oldest_key_time": 0, "file_creation_time": 1732398897, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294191, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:55:00.529671 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294192.sst) /dev/shm/ToplingTemp/Topling-284055-t6G4M6 2024/11/24-05:55:01.487737 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294192.sst): first pass time = 0.96's, 140.098'MB/sec 2024/11/24-05:55:02.817246 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294192.sst): old prealloc_size = 147639500, real_size = 31514016 2024/11/24-05:55:02.817827 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294192.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.12 2024/11/24-05:55:02.818613 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294192.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:55:03.632441 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294192.sst): second pass time = 1.33's, 81.640'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.661'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3165.128'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.12's, 287.031'MB/sec dict compress time = 0.00's, 1085010.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 28342.563'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.640'MB/sec zip pipeline throughput = 81.640'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.12 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0356 GB } UnZip/Zip{ index = 10.4443 value = 3.6743 dict = 1.00 all = 3.7711 } Zip/UnZip{ index = 0.0957 value = 0.2722 dict = 1.00 all = 0.2652 } ---------------------------- total value len = 2252.244169 GB avg = 0.200 KB (by entry num) total key len = 307.089824 GB avg = 0.027 KB total ukey len = 237.502486 GB avg = 0.015 KB total ukey num = 15.746049081 Billion total entry num = 11.287363029 Billion write speed all = 33.857467798 MB/sec (with seq num) write speed all = 32.662902239 MB/sec (without seq num) 2024/11/24-05:55:03.675940 139957349447232 [compaction_job.cc:1948] [default] [JOB 42486] Generated table #294192: 919300 keys, 35590706 bytes, temperature: kWarm 2024/11/24-05:55:03.676027 139957349447232 EVENT_LOG_v1 {"time_micros": 1732398903675989, "cf_name": "default", "job": 42486, "event": "table_file_creation", "file_number": 294192, "file_size": 35590706, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4310865210, "largest_seqno": 4462068628, "table_properties": {"data_size": 29523600, "index_size": 1760384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398900, "oldest_key_time": 0, "file_creation_time": 1732398900, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294192, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:55:03.714629 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294193.sst) /dev/shm/ToplingTemp/Topling-284055-R9HkU5 2024/11/24-05:55:04.746694 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294193.sst): first pass time = 1.03's, 130.052'MB/sec 2024/11/24-05:55:06.110665 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294193.sst): old prealloc_size = 147639500, real_size = 33224112 2024/11/24-05:55:06.111255 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294193.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927538 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.70 avg-zval = 33.67 2024/11/24-05:55:06.112880 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294193.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:55:06.965720 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294193.sst): second pass time = 1.36's, 79.454'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.653'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3113.411'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 298.261'MB/sec dict compress time = 0.00's, 1079942.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 28136.679'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.454'MB/sec zip pipeline throughput = 79.454'MB/sec entries = 927538 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.70 avg-zval = 33.67 usrkeys = 927538 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.70 avg-zval = 33.67 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927538 bytes = 3246400 } seq expand size = 6492766 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5123 value = 3.4664 dict = 1.00 all = 3.5960 } Zip/UnZip{ index = 0.0951 value = 0.2885 dict = 1.00 all = 0.2781 } ---------------------------- total value len = 2252.358909 GB avg = 0.200 KB (by entry num) total key len = 307.115795 GB avg = 0.027 KB total ukey len = 237.521036 GB avg = 0.015 KB total ukey num = 15.746976619 Billion total entry num = 11.288290567 Billion write speed all = 33.857836265 MB/sec (with seq num) write speed all = 32.663225220 MB/sec (without seq num) 2024/11/24-05:55:07.012322 139957349447232 [compaction_job.cc:1948] [default] [JOB 42486] Generated table #294193: 927538 keys, 37324438 bytes, temperature: kWarm 2024/11/24-05:55:07.012400 139957349447232 EVENT_LOG_v1 {"time_micros": 1732398907012360, "cf_name": "default", "job": 42486, "event": "table_file_creation", "file_number": 294193, "file_size": 37324438, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178686, "largest_seqno": 4467942834, "table_properties": {"data_size": 31227360, "index_size": 1764672, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25971064, "raw_average_key_size": 28, "raw_value_size": 108246724, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927538, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398903, "oldest_key_time": 0, "file_creation_time": 1732398903, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294193, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:55:07.050622 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294194.sst) /dev/shm/ToplingTemp/Topling-284055-dIC2a5 2024/11/24-05:55:08.058204 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294194.sst): first pass time = 1.01's, 133.213'MB/sec 2024/11/24-05:55:09.381628 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294194.sst): old prealloc_size = 147639500, real_size = 30817872 2024/11/24-05:55:09.382220 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294194.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926701 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0288 GB avg-val = 116.83 avg-zval = 31.09 2024/11/24-05:55:09.382964 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294194.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:55:10.222718 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294194.sst): second pass time = 1.32's, 81.890'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 41.374'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3114.683'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.324'MB/sec write SST data time = 0.13's, 276.529'MB/sec dict compress time = 0.00's, 1080606.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35676.516'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.890'MB/sec zip pipeline throughput = 81.890'MB/sec entries = 926701 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.83 avg-zval = 31.09 usrkeys = 926701 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.83 avg-zval = 31.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926701 bytes = 3127632 } seq expand size = 6486907 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0288 GB dict = 1.08 MB all = 0.0348 GB } UnZip/Zip{ index = 10.4437 value = 3.7579 dict = 1.00 all = 3.8570 } Zip/UnZip{ index = 0.0958 value = 0.2661 dict = 1.00 all = 0.2593 } ---------------------------- total value len = 2252.473666 GB avg = 0.200 KB (by entry num) total key len = 307.141743 GB avg = 0.027 KB total ukey len = 237.539570 GB avg = 0.015 KB total ukey num = 15.747903320 Billion total entry num = 11.289217268 Billion write speed all = 33.858238785 MB/sec (with seq num) write speed all = 32.663581142 MB/sec (without seq num) 2024/11/24-05:55:10.266742 139957349447232 [compaction_job.cc:1948] [default] [JOB 42486] Generated table #294194: 926701 keys, 34798382 bytes, temperature: kWarm 2024/11/24-05:55:10.266811 139957349447232 EVENT_LOG_v1 {"time_micros": 1732398910266776, "cf_name": "default", "job": 42486, "event": "table_file_creation", "file_number": 294194, "file_size": 34798382, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4360343923, "largest_seqno": 4467942844, "table_properties": {"data_size": 28811328, "index_size": 1774656, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25947628, "raw_average_key_size": 28, "raw_value_size": 108270110, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926701, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398907, "oldest_key_time": 0, "file_creation_time": 1732398907, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294194, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:55:10.307565 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294195.sst) /dev/shm/ToplingTemp/Topling-284055-gn3EG6 2024/11/24-05:55:11.295873 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294195.sst): first pass time = 0.99's, 135.811'MB/sec 2024/11/24-05:55:12.628841 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294195.sst): old prealloc_size = 147639500, real_size = 31717232 2024/11/24-05:55:12.629427 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294195.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920182 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1085 GB zip-val = 0.0297 GB avg-val = 117.86 avg-zval = 32.32 2024/11/24-05:55:12.630175 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294195.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:55:13.448086 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294195.sst): second pass time = 1.33's, 81.409'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.697'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3095.620'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.13's, 285.686'MB/sec dict compress time = 0.00's, 1084352.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 32253.183'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.409'MB/sec zip pipeline throughput = 81.409'MB/sec entries = 920182 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.86 avg-zval = 32.32 usrkeys = 920182 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.86 avg-zval = 32.32 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 920182 bytes = 3220656 } seq expand size = 6441274 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.08 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5140 value = 3.6471 dict = 1.00 all = 3.7493 } Zip/UnZip{ index = 0.0951 value = 0.2742 dict = 1.00 all = 0.2667 } ---------------------------- total value len = 2252.588560 GB avg = 0.200 KB (by entry num) total key len = 307.167508 GB avg = 0.027 KB total ukey len = 237.557974 GB avg = 0.015 KB total ukey num = 15.748823502 Billion total entry num = 11.290137450 Billion write speed all = 33.858654834 MB/sec (with seq num) write speed all = 32.663950786 MB/sec (without seq num) 2024/11/24-05:55:13.493344 139957349447232 [compaction_job.cc:1948] [default] [JOB 42486] Generated table #294195: 920182 keys, 35798032 bytes, temperature: kWarm 2024/11/24-05:55:13.493416 139957349447232 EVENT_LOG_v1 {"time_micros": 1732398913493380, "cf_name": "default", "job": 42486, "event": "table_file_creation", "file_number": 294195, "file_size": 35798032, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178700, "largest_seqno": 4467940093, "table_properties": {"data_size": 29736608, "index_size": 1750400, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25765096, "raw_average_key_size": 28, "raw_value_size": 108452712, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920182, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398910, "oldest_key_time": 0, "file_creation_time": 1732398910, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294195, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:55:13.534834 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294196.sst) /dev/shm/ToplingTemp/Topling-284055-itLTa7 2024/11/24-05:55:14.441492 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294196.sst): first pass time = 0.91's, 148.042'MB/sec 2024/11/24-05:55:15.770523 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294196.sst): old prealloc_size = 147639500, real_size = 32939952 2024/11/24-05:55:15.771101 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294196.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927648 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.69 avg-zval = 33.36 2024/11/24-05:55:15.771875 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294196.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:55:16.620393 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294196.sst): second pass time = 1.33's, 81.495'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.585'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3181.287'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 294.633'MB/sec dict compress time = 0.00's, 1080102.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36614.868'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.495'MB/sec zip pipeline throughput = 81.495'MB/sec entries = 927648 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.69 avg-zval = 33.36 usrkeys = 927648 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.69 avg-zval = 33.36 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927648 bytes = 3246784 } seq expand size = 6493536 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5128 value = 3.4982 dict = 1.00 all = 3.6235 } Zip/UnZip{ index = 0.0951 value = 0.2859 dict = 1.00 all = 0.2760 } ---------------------------- total value len = 2252.703297 GB avg = 0.200 KB (by entry num) total key len = 307.193482 GB avg = 0.027 KB total ukey len = 237.576527 GB avg = 0.015 KB total ukey num = 15.749751150 Billion total entry num = 11.291065098 Billion write speed all = 33.859095299 MB/sec (with seq num) write speed all = 32.664343222 MB/sec (without seq num) 2024/11/24-05:55:16.666170 139957349447232 [compaction_job.cc:1948] [default] [JOB 42486] Generated table #294196: 927648 keys, 37040822 bytes, temperature: kWarm 2024/11/24-05:55:16.666238 139957349447232 EVENT_LOG_v1 {"time_micros": 1732398916666203, "cf_name": "default", "job": 42486, "event": "table_file_creation", "file_number": 294196, "file_size": 37040822, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178719, "largest_seqno": 4467942855, "table_properties": {"data_size": 30943072, "index_size": 1764800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25974144, "raw_average_key_size": 28, "raw_value_size": 108243680, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927648, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398913, "oldest_key_time": 0, "file_creation_time": 1732398913, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294196, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:55:16.704279 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294197.sst) /dev/shm/ToplingTemp/Topling-284055-NXGYx5 2024/11/24-05:55:17.681286 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294197.sst): first pass time = 0.98's, 137.382'MB/sec 2024/11/24-05:55:18.993125 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294197.sst): old prealloc_size = 147639500, real_size = 31492048 2024/11/24-05:55:18.993653 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294197.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.09 2024/11/24-05:55:18.994404 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294197.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:55:19.809842 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294197.sst): second pass time = 1.31's, 82.745'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.552'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3477.614'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.13's, 283.620'MB/sec dict compress time = 0.00's, 1085010.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 33728.434'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.745'MB/sec zip pipeline throughput = 82.745'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.09 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0356 GB } UnZip/Zip{ index = 10.4439 value = 3.6770 dict = 1.00 all = 3.7735 } Zip/UnZip{ index = 0.0957 value = 0.2720 dict = 1.00 all = 0.2650 } ---------------------------- total value len = 2252.818209 GB avg = 0.200 KB (by entry num) total key len = 307.219223 GB avg = 0.027 KB total ukey len = 237.594913 GB avg = 0.015 KB total ukey num = 15.750670450 Billion total entry num = 11.291984398 Billion write speed all = 33.859527282 MB/sec (with seq num) write speed all = 32.664728334 MB/sec (without seq num) 2024/11/24-05:55:19.853446 139957349447232 [compaction_job.cc:1948] [default] [JOB 42486] Generated table #294197: 919300 keys, 35568754 bytes, temperature: kWarm 2024/11/24-05:55:19.853518 139957349447232 EVENT_LOG_v1 {"time_micros": 1732398919853482, "cf_name": "default", "job": 42486, "event": "table_file_creation", "file_number": 294197, "file_size": 35568754, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4305717291, "largest_seqno": 4457212384, "table_properties": {"data_size": 29501568, "index_size": 1760448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398916, "oldest_key_time": 0, "file_creation_time": 1732398916, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294197, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:55:19.896470 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294198.sst) /dev/shm/ToplingTemp/Topling-284055-Z7F244 2024/11/24-05:55:20.879413 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294198.sst): first pass time = 0.98's, 136.552'MB/sec 2024/11/24-05:55:22.195184 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294198.sst): old prealloc_size = 147639500, real_size = 33218704 2024/11/24-05:55:22.195712 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294198.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927933 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.64 avg-zval = 33.65 2024/11/24-05:55:22.196491 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294198.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:55:23.057495 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294198.sst): second pass time = 1.32's, 82.306'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.280'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3485.583'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.316'MB/sec write SST data time = 0.13's, 292.330'MB/sec dict compress time = 0.00's, 1079704.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36985.031'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.306'MB/sec zip pipeline throughput = 82.306'MB/sec entries = 927933 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.64 avg-zval = 33.65 usrkeys = 927933 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.64 avg-zval = 33.65 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927933 bytes = 3247776 } seq expand size = 6495531 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5130 value = 3.4667 dict = 1.00 all = 3.5964 } Zip/UnZip{ index = 0.0951 value = 0.2885 dict = 1.00 all = 0.2781 } ---------------------------- total value len = 2252.932941 GB avg = 0.199 KB (by entry num) total key len = 307.245205 GB avg = 0.027 KB total ukey len = 237.613472 GB avg = 0.015 KB total ukey num = 15.751598383 Billion total entry num = 11.292912331 Billion write speed all = 33.859933961 MB/sec (with seq num) write speed all = 32.665088153 MB/sec (without seq num) 2024/11/24-05:55:23.101478 139957349447232 [compaction_job.cc:1948] [default] [JOB 42486] Generated table #294198: 927933 keys, 37320040 bytes, temperature: kWarm 2024/11/24-05:55:23.101550 139957349447232 EVENT_LOG_v1 {"time_micros": 1732398923101512, "cf_name": "default", "job": 42486, "event": "table_file_creation", "file_number": 294198, "file_size": 37320040, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178730, "largest_seqno": 4467942867, "table_properties": {"data_size": 31221248, "index_size": 1765312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25982124, "raw_average_key_size": 28, "raw_value_size": 108235658, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927933, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398919, "oldest_key_time": 0, "file_creation_time": 1732398919, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294198, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:55:23.139554 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294199.sst) /dev/shm/ToplingTemp/Topling-284055-Y4ei74 2024/11/24-05:55:24.159015 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294199.sst): first pass time = 1.02's, 131.661'MB/sec 2024/11/24-05:55:25.478831 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294199.sst): old prealloc_size = 147639500, real_size = 30875840 2024/11/24-05:55:25.479412 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294199.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925411 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0289 GB avg-val = 117.04 avg-zval = 31.20 2024/11/24-05:55:25.480162 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294199.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:55:26.294202 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294199.sst): second pass time = 1.32's, 82.143'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.746'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3166.557'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 276.812'MB/sec dict compress time = 0.00's, 1080798.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33023.649'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.143'MB/sec zip pipeline throughput = 82.143'MB/sec entries = 925411 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.04 avg-zval = 31.20 usrkeys = 925411 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.04 avg-zval = 31.20 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925411 bytes = 3123280 } seq expand size = 6477877 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0349 GB } UnZip/Zip{ index = 10.4454 value = 3.7512 dict = 1.00 all = 3.8510 } Zip/UnZip{ index = 0.0957 value = 0.2666 dict = 1.00 all = 0.2597 } ---------------------------- total value len = 2253.047725 GB avg = 0.199 KB (by entry num) total key len = 307.271116 GB avg = 0.027 KB total ukey len = 237.631980 GB avg = 0.015 KB total ukey num = 15.752523794 Billion total entry num = 11.293837742 Billion write speed all = 33.860345274 MB/sec (with seq num) write speed all = 32.665452703 MB/sec (without seq num) 2024/11/24-05:55:26.338560 139957349447232 [compaction_job.cc:1948] [default] [JOB 42486] Generated table #294199: 925411 keys, 34852526 bytes, temperature: kWarm 2024/11/24-05:55:26.338632 139957349447232 EVENT_LOG_v1 {"time_micros": 1732398926338597, "cf_name": "default", "job": 42486, "event": "table_file_creation", "file_number": 294199, "file_size": 34852526, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4353047493, "largest_seqno": 4467942886, "table_properties": {"data_size": 28872368, "index_size": 1771904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25911508, "raw_average_key_size": 28, "raw_value_size": 108306318, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925411, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398923, "oldest_key_time": 0, "file_creation_time": 1732398923, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294199, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:55:26.387181 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294200.sst) /dev/shm/ToplingTemp/Topling-284055-X330b7 2024/11/24-05:55:27.360493 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294200.sst): first pass time = 0.97's, 137.904'MB/sec 2024/11/24-05:55:28.688143 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294200.sst): old prealloc_size = 147639500, real_size = 32312288 2024/11/24-05:55:28.688685 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294200.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921422 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0303 GB avg-val = 117.66 avg-zval = 32.92 2024/11/24-05:55:28.690088 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294200.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:55:29.511914 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294200.sst): second pass time = 1.33's, 81.708'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 44.184'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3358.013'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 289.355'MB/sec dict compress time = 0.00's, 1083702.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 35305.489'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.708'MB/sec zip pipeline throughput = 81.708'MB/sec entries = 921422 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.66 avg-zval = 32.92 usrkeys = 921422 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.66 avg-zval = 32.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921422 bytes = 3224992 } seq expand size = 6449954 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5131 value = 3.5747 dict = 1.00 all = 3.6877 } Zip/UnZip{ index = 0.0951 value = 0.2797 dict = 1.00 all = 0.2712 } ---------------------------- total value len = 2253.162593 GB avg = 0.199 KB (by entry num) total key len = 307.296916 GB avg = 0.027 KB total ukey len = 237.650408 GB avg = 0.015 KB total ukey num = 15.753445216 Billion total entry num = 11.294759164 Billion write speed all = 33.860764687 MB/sec (with seq num) write speed all = 32.665825479 MB/sec (without seq num) 2024/11/24-05:55:29.556398 139957349447232 [compaction_job.cc:1948] [default] [JOB 42486] Generated table #294200: 921422 keys, 36396422 bytes, temperature: kWarm 2024/11/24-05:55:29.556474 139957349447232 EVENT_LOG_v1 {"time_micros": 1732398929556433, "cf_name": "default", "job": 42486, "event": "table_file_creation", "file_number": 294200, "file_size": 36396422, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680186, "largest_seqno": 4467941959, "table_properties": {"data_size": 30328848, "index_size": 1752896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25799816, "raw_average_key_size": 28, "raw_value_size": 108418032, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921422, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398926, "oldest_key_time": 0, "file_creation_time": 1732398926, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294200, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:55:29.595810 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294201.sst) /dev/shm/ToplingTemp/Topling-284055-F4BGv6 2024/11/24-05:55:30.478195 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294201.sst): first pass time = 0.88's, 152.115'MB/sec 2024/11/24-05:55:31.835764 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294201.sst): old prealloc_size = 147639500, real_size = 32788064 2024/11/24-05:55:31.836313 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294201.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927659 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0308 GB avg-val = 116.68 avg-zval = 33.19 2024/11/24-05:55:31.837093 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294201.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:55:32.690451 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294201.sst): second pass time = 1.36's, 79.777'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.162'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3297.196'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 292.715'MB/sec dict compress time = 0.00's, 1080182.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 37777.848'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.777'MB/sec zip pipeline throughput = 79.777'MB/sec entries = 927659 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.68 avg-zval = 33.19 usrkeys = 927659 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.68 avg-zval = 33.19 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927659 bytes = 3246816 } seq expand size = 6493613 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5144 value = 3.5154 dict = 1.00 all = 3.6384 } Zip/UnZip{ index = 0.0951 value = 0.2845 dict = 1.00 all = 0.2748 } ---------------------------- total value len = 2253.277330 GB avg = 0.199 KB (by entry num) total key len = 307.322890 GB avg = 0.027 KB total ukey len = 237.668962 GB avg = 0.015 KB total ukey num = 15.754372875 Billion total entry num = 11.295686823 Billion write speed all = 33.861202181 MB/sec (with seq num) write speed all = 32.666215061 MB/sec (without seq num) 2024/11/24-05:55:32.735422 139957349447232 [compaction_job.cc:1948] [default] [JOB 42486] Generated table #294201: 927659 keys, 36888966 bytes, temperature: kWarm 2024/11/24-05:55:32.735493 139957349447232 EVENT_LOG_v1 {"time_micros": 1732398932735457, "cf_name": "default", "job": 42486, "event": "table_file_creation", "file_number": 294201, "file_size": 36888966, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680199, "largest_seqno": 4467942898, "table_properties": {"data_size": 30791440, "index_size": 1764544, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25974452, "raw_average_key_size": 28, "raw_value_size": 108243418, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927659, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398929, "oldest_key_time": 0, "file_creation_time": 1732398929, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294201, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:55:32.771850 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294202.sst) /dev/shm/ToplingTemp/Topling-284055-IPcGW6 2024/11/24-05:55:33.738302 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294202.sst): first pass time = 0.97's, 138.882'MB/sec 2024/11/24-05:55:35.045039 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294202.sst): old prealloc_size = 147639500, real_size = 31439440 2024/11/24-05:55:35.045611 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294202.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 32.02 2024/11/24-05:55:35.046383 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294202.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:55:35.858336 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294202.sst): second pass time = 1.31's, 83.059'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.671'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3240.351'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.12's, 284.556'MB/sec dict compress time = 0.00's, 1085010.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 35777.030'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 83.059'MB/sec zip pipeline throughput = 83.059'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 32.02 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 32.02 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.3768 value = 3.6850 dict = 1.00 all = 3.7791 } Zip/UnZip{ index = 0.0964 value = 0.2714 dict = 1.00 all = 0.2646 } ---------------------------- total value len = 2253.392242 GB avg = 0.199 KB (by entry num) total key len = 307.348631 GB avg = 0.027 KB total ukey len = 237.687348 GB avg = 0.015 KB total ukey num = 15.755292175 Billion total entry num = 11.296606123 Billion write speed all = 33.861643644 MB/sec (with seq num) write speed all = 32.666609333 MB/sec (without seq num) 2024/11/24-05:55:35.902138 139957349447232 [compaction_job.cc:1948] [default] [JOB 42486] Generated table #294202: 919300 keys, 35516146 bytes, temperature: kWarm 2024/11/24-05:55:35.902213 139957349447232 EVENT_LOG_v1 {"time_micros": 1732398935902177, "cf_name": "default", "job": 42486, "event": "table_file_creation", "file_number": 294202, "file_size": 35516146, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4307101885, "largest_seqno": 4458407486, "table_properties": {"data_size": 29437568, "index_size": 1771840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398932, "oldest_key_time": 0, "file_creation_time": 1732398932, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294202, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:55:35.941973 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294203.sst) /dev/shm/ToplingTemp/Topling-284055-CLzcp5 2024/11/24-05:55:36.917608 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294203.sst): first pass time = 0.98's, 137.576'MB/sec 2024/11/24-05:55:38.249927 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294203.sst): old prealloc_size = 147639500, real_size = 33235968 2024/11/24-05:55:38.250524 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294203.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927339 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 116.73 avg-zval = 33.69 2024/11/24-05:55:38.251300 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294203.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:55:39.104543 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294203.sst): second pass time = 1.33's, 81.298'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.688'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3067.959'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 295.503'MB/sec dict compress time = 0.00's, 1080194.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36660.241'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.298'MB/sec zip pipeline throughput = 81.298'MB/sec entries = 927339 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.73 avg-zval = 33.69 usrkeys = 927339 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.73 avg-zval = 33.69 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927339 bytes = 3245696 } seq expand size = 6491373 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5154 value = 3.4652 dict = 1.00 all = 3.5949 } Zip/UnZip{ index = 0.0951 value = 0.2886 dict = 1.00 all = 0.2782 } ---------------------------- total value len = 2253.506986 GB avg = 0.199 KB (by entry num) total key len = 307.374596 GB avg = 0.027 KB total ukey len = 237.705894 GB avg = 0.015 KB total ukey num = 15.756219514 Billion total entry num = 11.297533462 Billion write speed all = 33.862050735 MB/sec (with seq num) write speed all = 32.666969623 MB/sec (without seq num) 2024/11/24-05:55:39.150562 139957349447232 [compaction_job.cc:1948] [default] [JOB 42486] Generated table #294203: 927339 keys, 37335890 bytes, temperature: kWarm 2024/11/24-05:55:39.150632 139957349447232 EVENT_LOG_v1 {"time_micros": 1732398939150597, "cf_name": "default", "job": 42486, "event": "table_file_creation", "file_number": 294203, "file_size": 37335890, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680212, "largest_seqno": 4467942911, "table_properties": {"data_size": 31240176, "index_size": 1763776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25965492, "raw_average_key_size": 28, "raw_value_size": 108252374, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927339, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398935, "oldest_key_time": 0, "file_creation_time": 1732398935, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294203, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:55:39.195637 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294204.sst) /dev/shm/ToplingTemp/Topling-284055-pTJYK5 2024/11/24-05:55:40.124051 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294204.sst): first pass time = 0.93's, 144.573'MB/sec 2024/11/24-05:55:41.429708 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294204.sst): old prealloc_size = 147639500, real_size = 30678576 2024/11/24-05:55:41.430253 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294204.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926972 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0287 GB avg-val = 116.79 avg-zval = 30.93 2024/11/24-05:55:41.431063 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294204.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:55:42.248681 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294204.sst): second pass time = 1.30's, 82.974'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.322'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3445.767'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 268.697'MB/sec dict compress time = 0.00's, 1080398.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34986.982'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 82.974'MB/sec zip pipeline throughput = 82.974'MB/sec entries = 926972 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.79 avg-zval = 30.93 usrkeys = 926972 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.79 avg-zval = 30.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926972 bytes = 3128544 } seq expand size = 6488804 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0287 GB dict = 1.08 MB all = 0.0347 GB } UnZip/Zip{ index = 10.4441 value = 3.7760 dict = 1.00 all = 3.8724 } Zip/UnZip{ index = 0.0957 value = 0.2648 dict = 1.00 all = 0.2582 } ---------------------------- total value len = 2253.621737 GB avg = 0.199 KB (by entry num) total key len = 307.400552 GB avg = 0.027 KB total ukey len = 237.724434 GB avg = 0.015 KB total ukey num = 15.757146486 Billion total entry num = 11.298460434 Billion write speed all = 33.862503463 MB/sec (with seq num) write speed all = 32.667373980 MB/sec (without seq num) 2024/11/24-05:55:42.294794 139957349447232 [compaction_job.cc:1948] [default] [JOB 42486] Generated table #294204: 926972 keys, 34659742 bytes, temperature: kWarm 2024/11/24-05:55:42.294865 139957349447232 EVENT_LOG_v1 {"time_micros": 1732398942294829, "cf_name": "default", "job": 42486, "event": "table_file_creation", "file_number": 294204, "file_size": 34659742, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4361918535, "largest_seqno": 4467942923, "table_properties": {"data_size": 28671520, "index_size": 1775104, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25955216, "raw_average_key_size": 28, "raw_value_size": 108262548, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926972, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398939, "oldest_key_time": 0, "file_creation_time": 1732398939, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294204, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:55:42.332138 139957349447232 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294205.sst) /dev/shm/ToplingTemp/Topling-284055-3j1y37 2024/11/24-05:55:42.563338 139957349447232 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294205.sst): first pass time = 0.23's, 243.501'MB/sec 2024/11/24-05:55:43.145803 139957349447232 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294205.sst): old prealloc_size = 147639500, real_size = 15194288 2024/11/24-05:55:43.146166 139957349447232 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294205.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 386352 prefix = 4 raw-key = 0.0062 GB zip-key = 0.0007 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0455 GB zip-val = 0.0144 GB avg-val = 117.69 avg-zval = 37.16 2024/11/24-05:55:43.146559 139957349447232 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294205.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0009 GB, waited 0.000 sec, Key+Value = 0.056 GB 2024/11/24-05:55:43.499955 139957349447232 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00/294205.sst): second pass time = 0.58's, 78.170'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.35's, 47.769'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2162.075'MB/sec (index lex order gen) rebuild zvType time = 0.30's, 0.321'MB/sec write SST data time = 0.05's, 318.243'MB/sec dict compress time = 0.00's, 461750.000'MB/sec z-dict build time = 0.00's, sample length = 0.462'MB, throughput = 13125.355'MB/sec zip my value time = 0.58's, unzip length = 0.045'GB zip my value throughput = 78.170'MB/sec zip pipeline throughput = 78.170'MB/sec entries = 386352 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.69 avg-zval = 37.16 usrkeys = 386352 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.69 avg-zval = 37.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 386352 bytes = 1352240 } seq expand size = 2704464 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0077 GB value = 0.0455 GB dict = 0.46 MB all = 0.0563 GB } __ZipSize{ index = 0.0007 GB value = 0.0144 GB dict = 0.46 MB all = 0.0169 GB } UnZip/Zip{ index = 10.4334 value = 3.1672 dict = 1.00 all = 3.3275 } Zip/UnZip{ index = 0.0958 value = 0.3157 dict = 1.00 all = 0.3005 } ---------------------------- total value len = 2253.669913 GB avg = 0.199 KB (by entry num) total key len = 307.411369 GB avg = 0.027 KB total ukey len = 237.732161 GB avg = 0.015 KB total ukey num = 15.757532838 Billion total entry num = 11.298846786 Billion write speed all = 33.862723238 MB/sec (with seq num) write speed all = 32.667572661 MB/sec (without seq num) 2024/11/24-05:55:43.525083 139957349447232 [compaction_job.cc:1948] [default] [JOB 42486] Generated table #294205: 386352 keys, 16916357 bytes, temperature: kWarm 2024/11/24-05:55:43.525150 139957349447232 EVENT_LOG_v1 {"time_micros": 1732398943525116, "cf_name": "default", "job": 42486, "event": "table_file_creation", "file_number": 294205, "file_size": 16916357, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4467942244, "table_properties": {"data_size": 14356896, "index_size": 740608, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10817856, "raw_average_key_size": 28, "raw_value_size": 45470968, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 386352, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398942, "oldest_key_time": 0, "file_creation_time": 1732398942, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294205, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:55:43.542820 139957349447232 [compaction_job.cc:703] job-42486: subcompact[0], size: 0.739319 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42486/att-00 [294185,294186,294187,294188,294189,294190,294191,294192,294193,294194,294195,294196,294197,294198,294199,294200,294201,294202,294203,294204,294205] 2024/11/24-05:55:43.666736 139957349447232 (Original Log Time 2024/11/24-05:54:37.583418) EVENT_LOG_v1 {"time_micros": 1732398877583392, "job": 42486, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [294120], "files_L3": [294157, 294087, 294088, 294089, 294090, 294091, 294092, 294093, 294094, 294095, 294044, 294045, 294046, 294047, 294048, 294049, 294050, 294051, 294052, 294053, 294054, 294158], "score": 0.921233, "input_data_size": 764079190, "oldest_snapshot_seqno": 4468407137} 2024/11/24-05:55:43.666740 139957349447232 (Original Log Time 2024/11/24-05:55:43.664494) [compaction_job.cc:2022] [default] [JOB 42486] Compacted 1@2 + 22@3 files to L3 => 739319449 bytes 2024/11/24-05:55:43.666741 139957349447232 (Original Log Time 2024/11/24-05:55:43.666044) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 21 0 0 0] max score 0.03, MB/sec: 11.6 rd, 11.2 wr, level 3, files in(1, 22) out(21 +0 blob) MB in(27.1, 701.6 +0.0 blob) out(705.1 +0.0 blob), read-write-amplify(52.9) write-amplify(26.0) OK, records in: 18982085, records dropped: 110410 output_compression: Snappy 2024/11/24-05:55:43.666745 139957349447232 (Original Log Time 2024/11/24-05:55:43.666707) EVENT_LOG_v1 {"time_micros": 1732398943666068, "cf": "default", "job": 42486, "event": "compaction_finished", "compaction_time_micros": 65959410, "compaction_time_cpu_micros": 27997956, "output_level": 3, "num_output_files": 21, "total_output_size": 739319449, "num_input_records": 18982085, "num_output_records": 18871675, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 518290589, "file_range_sync_nanos": 0, "file_fsync_nanos": 14418, "file_prepare_write_nanos": 50363, "lsm_state": [0, 0, 0, 21, 0, 0, 0]} 2024/11/24-05:55:43.804527 139957349447232 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732398877,"executesMs":66208,"compactionJobId":42486,"attempt":0,"compactionInputRawBytes":2756323118,"compactionInputZipBytes":764079190,"compactionOutputRawBytes":0,"compactionOutputZipBytes":739319449,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-ab2d4da63ad3ed280d2e7a4e6bc33781-bc37a1b19d05f679-00"}}, response = 2024/11/24-05:55:43.804620 139957349447232 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-42486/att-00: olev 3, work 66.209 s, result 7.745 ms, install 2.268 ms, input{raw 2.567 GiB zip 728.683 MiB}