2024/11/22-23:16:42.550928 140025003607616 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00: fork to child time = 0.009108 sec 2024/11/22-23:16:42.551900 140025003607616 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00 2024/11/22-23:16:42.558595 140025003607616 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00 2024/11/22-23:16:42.559716 140025003607616 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-23:16:42.560002 140025003607616 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-23:16:42.611185 140025003607616 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-13761/att-00/MANIFEST-206347 2024/11/22-23:16:42.642158 140025003607616 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-13761/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 233387, last_sequence is 3636588824, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 233219 2024/11/22-23:16:42.642167 140025003607616 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-23:16:42.642275 140025003607616 [version_set.cc:5721] Creating manifest 233387 2024/11/22-23:16:42.678675 140025003607616 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-23:16:42.699307 140025003607616 [compaction_job.cc:2437] [default] [JOB 13761] Compacting 1@2 + 12@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/22-23:16:42.699322 140025003607616 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 10329 Base level 2, inputs: [233328(27MB)], [233384(37MB) 233385(5459KB) 233266(36MB) 233267(35MB) 233268(6992KB) 233223(35MB) 233224(35MB) 233225(33MB) 233226(36MB) 233227(36MB) 233228(36MB) 233357(35MB)] 2024/11/22-23:16:42.706669 140025003607616 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-23:16:42.706702 140025003607616 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-23:16:42.706722 140025003607616 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-23:16:42.706728 140025003607616 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-23:16:42.706738 140025003607616 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-23:16:43.231740 140025003607616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233388.sst) /dev/shm/ToplingTemp/Topling-329614-ZEF4f5 2024/11/22-23:16:43.774424 140025003607616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233388.sst): first pass time = 0.54's, 247.348'MB/sec 2024/11/22-23:16:45.108430 140025003607616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233388.sst): old prealloc_size = 147639500, real_size = 35002928 2024/11/22-23:16:45.109016 140025003607616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233388.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931063 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0330 GB avg-val = 116.16 avg-zval = 35.42 2024/11/22-23:16:45.109837 140025003607616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233388.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:16:45.994693 140025003607616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233388.sst): second pass time = 1.33's, 81.152'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 44.264'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3171.145'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.308'MB/sec write SST data time = 0.13's, 302.933'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33428.403'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.152'MB/sec zip pipeline throughput = 81.152'MB/sec entries = 931063 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.16 avg-zval = 35.42 usrkeys = 931063 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.16 avg-zval = 35.42 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931063 bytes = 3375120 } seq expand size = 6517441 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0330 GB dict = 1.08 MB all = 0.0392 GB } UnZip/Zip{ index = 10.3743 value = 3.2797 dict = 1.00 all = 3.4214 } Zip/UnZip{ index = 0.0964 value = 0.3049 dict = 1.00 all = 0.2923 } ---------------------------- total value len = 0.114666 GB avg = 0.123 KB (by entry num) total key len = 0.026070 GB avg = 0.028 KB total ukey len = 0.018621 GB avg = 0.020 KB total ukey num = 0.000931063 Billion total entry num = 0.000931063 Billion write speed all = 0.002713602 MB/sec (with seq num) write speed all = 0.002569983 MB/sec (without seq num) 2024/11/22-23:16:46.028095 140025003607616 [compaction_job.cc:1948] [default] [JOB 13761] Generated table #233388: 931063 keys, 39229420 bytes, temperature: kWarm 2024/11/22-23:16:46.028191 140025003607616 EVENT_LOG_v1 {"time_micros": 1732288606028130, "cf_name": "default", "job": 13761, "event": "table_file_creation", "file_number": 233388, "file_size": 39229420, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701386, "largest_seqno": 3636042636, "table_properties": {"data_size": 32975008, "index_size": 1794944, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26069764, "raw_average_key_size": 28, "raw_value_size": 108148086, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931063, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288603, "oldest_key_time": 0, "file_creation_time": 1732288603, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 233388, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:16:46.065268 140025003607616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233389.sst) /dev/shm/ToplingTemp/Topling-329614-kjVSg5 2024/11/22-23:16:47.050237 140025003607616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233389.sst): first pass time = 0.98's, 136.271'MB/sec 2024/11/22-23:16:48.390083 140025003607616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233389.sst): old prealloc_size = 147639500, real_size = 34690448 2024/11/22-23:16:48.390652 140025003607616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233389.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926395 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0327 GB avg-val = 116.88 avg-zval = 35.29 2024/11/22-23:16:48.391444 140025003607616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233389.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:16:49.294237 140025003607616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233389.sst): second pass time = 1.34's, 80.902'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 42.900'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3213.764'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.298'MB/sec write SST data time = 0.13's, 304.277'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33053.472'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.902'MB/sec zip pipeline throughput = 80.902'MB/sec entries = 926395 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.88 avg-zval = 35.29 usrkeys = 926395 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.88 avg-zval = 35.29 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926395 bytes = 3242400 } seq expand size = 6484765 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0327 GB dict = 1.08 MB all = 0.0388 GB } UnZip/Zip{ index = 10.5161 value = 3.3116 dict = 1.00 all = 3.4603 } Zip/UnZip{ index = 0.0951 value = 0.3020 dict = 1.00 all = 0.2890 } ---------------------------- total value len = 0.229429 GB avg = 0.124 KB (by entry num) total key len = 0.052009 GB avg = 0.028 KB total ukey len = 0.037149 GB avg = 0.020 KB total ukey num = 0.001857458 Billion total entry num = 0.001857458 Billion write speed all = 0.005426228 MB/sec (with seq num) write speed all = 0.005139728 MB/sec (without seq num) 2024/11/22-23:16:49.327407 140025003607616 [compaction_job.cc:1948] [default] [JOB 13761] Generated table #233389: 926395 keys, 38787606 bytes, temperature: kWarm 2024/11/22-23:16:49.327468 140025003607616 EVENT_LOG_v1 {"time_micros": 1732288609327436, "cf_name": "default", "job": 13761, "event": "table_file_creation", "file_number": 233389, "file_size": 38787606, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3636042860, "table_properties": {"data_size": 32696832, "index_size": 1761856, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25939060, "raw_average_key_size": 28, "raw_value_size": 108278718, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926395, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288606, "oldest_key_time": 0, "file_creation_time": 1732288606, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 233389, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:16:49.362592 140025003607616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233390.sst) /dev/shm/ToplingTemp/Topling-329614-Obd8U3 2024/11/22-23:16:50.163044 140025003607616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233390.sst): first pass time = 0.80's, 167.685'MB/sec 2024/11/22-23:16:51.517540 140025003607616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233390.sst): old prealloc_size = 147639500, real_size = 33225904 2024/11/22-23:16:51.518126 140025003607616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233390.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931675 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0312 GB avg-val = 116.06 avg-zval = 33.50 2024/11/22-23:16:51.518859 140025003607616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233390.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:16:52.441489 140025003607616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233390.sst): second pass time = 1.35's, 79.869'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.285'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3161.310'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.293'MB/sec write SST data time = 0.13's, 291.312'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33275.248'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 79.869'MB/sec zip pipeline throughput = 79.869'MB/sec entries = 931675 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.06 avg-zval = 33.50 usrkeys = 931675 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.06 avg-zval = 33.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931675 bytes = 3144416 } seq expand size = 6521725 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.4433 value = 3.4648 dict = 1.00 all = 3.6060 } Zip/UnZip{ index = 0.0958 value = 0.2886 dict = 1.00 all = 0.2773 } ---------------------------- total value len = 0.344082 GB avg = 0.123 KB (by entry num) total key len = 0.078096 GB avg = 0.028 KB total ukey len = 0.055783 GB avg = 0.020 KB total ukey num = 0.002789133 Billion total entry num = 0.002789133 Billion write speed all = 0.008139246 MB/sec (with seq num) write speed all = 0.007709067 MB/sec (without seq num) 2024/11/22-23:16:52.474057 140025003607616 [compaction_job.cc:1948] [default] [JOB 13761] Generated table #233390: 931675 keys, 37220936 bytes, temperature: kWarm 2024/11/22-23:16:52.474124 140025003607616 EVENT_LOG_v1 {"time_micros": 1732288612474094, "cf_name": "default", "job": 13761, "event": "table_file_creation", "file_number": 233390, "file_size": 37220936, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3504428200, "largest_seqno": 3636040740, "table_properties": {"data_size": 31208544, "index_size": 1784256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26086900, "raw_average_key_size": 28, "raw_value_size": 108130886, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931675, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288609, "oldest_key_time": 0, "file_creation_time": 1732288609, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 233390, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:16:52.509887 140025003607616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233391.sst) /dev/shm/ToplingTemp/Topling-329614-dBu9f6 2024/11/22-23:16:53.375704 140025003607616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233391.sst): first pass time = 0.87's, 155.025'MB/sec 2024/11/22-23:16:54.702096 140025003607616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233391.sst): old prealloc_size = 147639500, real_size = 33231008 2024/11/22-23:16:54.702660 140025003607616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233391.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932269 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0312 GB avg-val = 115.97 avg-zval = 33.49 2024/11/22-23:16:54.703374 140025003607616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233391.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:16:55.631738 140025003607616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233391.sst): second pass time = 1.33's, 81.585'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 39.827'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3259.029'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.292'MB/sec write SST data time = 0.13's, 281.559'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33765.716'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.585'MB/sec zip pipeline throughput = 81.585'MB/sec entries = 932269 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.97 avg-zval = 33.49 usrkeys = 932269 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.42 avg-zval = 33.49 TagRS{ kinds = RS_Key0_TagN bytes = 145760 } TagArray{ size = 859384 bytes = 2793008 } seq expand size = 6015688 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5167 value = 3.4624 dict = 1.00 all = 3.6251 } Zip/UnZip{ index = 0.0951 value = 0.2888 dict = 1.00 all = 0.2759 } ---------------------------- total value len = 0.458212 GB avg = 0.123 KB (by entry num) total key len = 0.104199 GB avg = 0.028 KB total ukey len = 0.074428 GB avg = 0.020 KB total ukey num = 0.003721402 Billion total entry num = 0.003721402 Billion write speed all = 0.010842170 MB/sec (with seq num) write speed all = 0.010268240 MB/sec (without seq num) 2024/11/22-23:16:55.665353 140025003607616 [compaction_job.cc:1948] [default] [JOB 13761] Generated table #233391: 932269 keys, 37024294 bytes, temperature: kWarm 2024/11/22-23:16:55.665415 140025003607616 EVENT_LOG_v1 {"time_micros": 1732288615665383, "cf_name": "default", "job": 13761, "event": "table_file_creation", "file_number": 233391, "file_size": 37024294, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3636042212, "table_properties": {"data_size": 31224848, "index_size": 1772928, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26103532, "raw_average_key_size": 28, "raw_value_size": 108114290, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932269, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288612, "oldest_key_time": 0, "file_creation_time": 1732288612, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 233391, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:16:55.701335 140025003607616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233392.sst) /dev/shm/ToplingTemp/Topling-329614-NRVF32 2024/11/22-23:16:56.558303 140025003607616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233392.sst): first pass time = 0.86's, 156.626'MB/sec 2024/11/22-23:16:57.957851 140025003607616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233392.sst): old prealloc_size = 147639500, real_size = 34931392 2024/11/22-23:16:57.958472 140025003607616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233392.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1081487 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1039 GB zip-val = 0.0326 GB avg-val = 96.10 avg-zval = 30.15 2024/11/22-23:16:57.958523 140025003607616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233392.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:16:58.583772 140025003607616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233392.sst): second pass time = 1.40's, 74.302'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.63's, 57.046'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3417.452'MB/sec (index lex order gen) rebuild zvType time = 0.49's, 0.557'MB/sec write SST data time = 0.14's, 254.554'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.033'MB, throughput = 33997.039'MB/sec zip my value time = 1.40's, unzip length = 0.104'GB zip my value throughput = 74.302'MB/sec zip pipeline throughput = 74.302'MB/sec entries = 1081487 avg-key = 28.00 avg-zkey = 1.90 avg-val = 96.10 avg-zval = 30.15 usrkeys = 1081487 avg-key = 20.00 avg-zkey = 1.90 avg-val = 96.10 avg-zval = 30.15 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 10 UnZipSize{ index = 0.0216 GB value = 0.1039 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0326 GB dict = 1.03 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5157 value = 3.1878 dict = 1.00 all = 3.7590 } Zip/UnZip{ index = 0.0951 value = 0.3137 dict = 1.00 all = 0.2660 } ---------------------------- total value len = 0.562148 GB avg = 0.117 KB (by entry num) total key len = 0.134481 GB avg = 0.028 KB total ukey len = 0.096058 GB avg = 0.020 KB total ukey num = 0.004802889 Billion total entry num = 0.004802889 Billion write speed all = 0.013428859 MB/sec (with seq num) write speed all = 0.012688180 MB/sec (without seq num) 2024/11/22-23:16:58.615730 140025003607616 [compaction_job.cc:1948] [default] [JOB 13761] Generated table #233392: 1081487 keys, 35705478 bytes, temperature: kWarm 2024/11/22-23:16:58.615796 140025003607616 EVENT_LOG_v1 {"time_micros": 1732288618615759, "cf_name": "default", "job": 13761, "event": "table_file_creation", "file_number": 233392, "file_size": 35705478, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 32603952, "index_size": 2056896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30281636, "raw_average_key_size": 28, "raw_value_size": 103936131, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1081487, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288615, "oldest_key_time": 0, "file_creation_time": 1732288615, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 233392, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:16:58.656321 140025003607616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233393.sst) /dev/shm/ToplingTemp/Topling-329614-McYNE3 2024/11/22-23:16:59.512443 140025003607616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233393.sst): first pass time = 0.86's, 156.780'MB/sec 2024/11/22-23:17:00.836579 140025003607616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233393.sst): old prealloc_size = 147639500, real_size = 33893552 2024/11/22-23:17:00.837141 140025003607616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233393.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921341 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0319 GB avg-val = 117.68 avg-zval = 34.62 2024/11/22-23:17:00.837760 140025003607616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233393.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:17:01.713491 140025003607616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233393.sst): second pass time = 1.32's, 81.949'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 42.641'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3265.008'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.309'MB/sec write SST data time = 0.13's, 284.710'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 35144.535'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.949'MB/sec zip pipeline throughput = 81.949'MB/sec entries = 921341 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.68 avg-zval = 34.62 usrkeys = 921341 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.90 avg-zval = 34.62 TagRS{ kinds = RS_Key0_TagN bytes = 144040 } TagArray{ size = 687310 bytes = 2491520 } seq expand size = 4811170 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4447 value = 3.3989 dict = 1.00 all = 3.5895 } Zip/UnZip{ index = 0.0957 value = 0.2942 dict = 1.00 all = 0.2786 } ---------------------------- total value len = 0.675379 GB avg = 0.118 KB (by entry num) total key len = 0.160278 GB avg = 0.028 KB total ukey len = 0.114485 GB avg = 0.020 KB total ukey num = 0.005724230 Billion total entry num = 0.005724230 Billion write speed all = 0.016107937 MB/sec (with seq num) write speed all = 0.015225226 MB/sec (without seq num) 2024/11/22-23:17:01.748640 140025003607616 [compaction_job.cc:1948] [default] [JOB 13761] Generated table #233393: 921341 keys, 37391554 bytes, temperature: kWarm 2024/11/22-23:17:01.748706 140025003607616 EVENT_LOG_v1 {"time_micros": 1732288621748672, "cf_name": "default", "job": 13761, "event": "table_file_creation", "file_number": 233393, "file_size": 37391554, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3565070457, "table_properties": {"data_size": 31898784, "index_size": 1764224, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25797548, "raw_average_key_size": 28, "raw_value_size": 108420186, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921341, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288618, "oldest_key_time": 0, "file_creation_time": 1732288618, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 233393, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:17:01.797214 140025003607616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233394.sst) /dev/shm/ToplingTemp/Topling-329614-txmoD3 2024/11/22-23:17:02.597802 140025003607616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233394.sst): first pass time = 0.80's, 167.658'MB/sec 2024/11/22-23:17:03.928964 140025003607616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233394.sst): old prealloc_size = 147639500, real_size = 33885456 2024/11/22-23:17:03.929490 140025003607616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233394.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927247 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0319 GB avg-val = 116.75 avg-zval = 34.38 2024/11/22-23:17:03.930268 140025003607616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233394.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:17:04.843754 140025003607616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233394.sst): second pass time = 1.33's, 81.365'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 41.524'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3510.371'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.295'MB/sec write SST data time = 0.13's, 293.020'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35346.287'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.365'MB/sec zip pipeline throughput = 81.365'MB/sec entries = 927247 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.75 avg-zval = 34.38 usrkeys = 927247 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.75 avg-zval = 34.38 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927247 bytes = 3245376 } seq expand size = 6490729 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.4442 value = 3.3959 dict = 1.00 all = 3.5334 } Zip/UnZip{ index = 0.0957 value = 0.2945 dict = 1.00 all = 0.2830 } ---------------------------- total value len = 0.790125 GB avg = 0.119 KB (by entry num) total key len = 0.186241 GB avg = 0.028 KB total ukey len = 0.133030 GB avg = 0.020 KB total ukey num = 0.006651477 Billion total entry num = 0.006651477 Billion write speed all = 0.018819067 MB/sec (with seq num) write speed all = 0.017793431 MB/sec (without seq num) 2024/11/22-23:17:04.881367 140025003607616 [compaction_job.cc:1948] [default] [JOB 13761] Generated table #233394: 927247 keys, 37985448 bytes, temperature: kWarm 2024/11/22-23:17:04.881434 140025003607616 EVENT_LOG_v1 {"time_micros": 1732288624881400, "cf_name": "default", "job": 13761, "event": "table_file_creation", "file_number": 233394, "file_size": 37985448, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3375334738, "largest_seqno": 3636042749, "table_properties": {"data_size": 31877824, "index_size": 1775616, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25962916, "raw_average_key_size": 28, "raw_value_size": 108254934, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927247, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288621, "oldest_key_time": 0, "file_creation_time": 1732288621, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 233394, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:17:04.925221 140025003607616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233395.sst) /dev/shm/ToplingTemp/Topling-329614-PYsD23 2024/11/22-23:17:05.738451 140025003607616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233395.sst): first pass time = 0.81's, 165.052'MB/sec 2024/11/22-23:17:07.079820 140025003607616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233395.sst): old prealloc_size = 147639500, real_size = 33122912 2024/11/22-23:17:07.080401 140025003607616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233395.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928412 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.57 avg-zval = 33.51 2024/11/22-23:17:07.081203 140025003607616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233395.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:17:08.015027 140025003607616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233395.sst): second pass time = 1.34's, 80.725'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.929'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3182.545'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.288'MB/sec write SST data time = 0.13's, 287.584'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 29289.838'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.725'MB/sec zip pipeline throughput = 80.725'MB/sec entries = 928412 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.57 avg-zval = 33.51 usrkeys = 928412 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.57 avg-zval = 33.51 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928412 bytes = 3365504 } seq expand size = 6498884 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4449 value = 3.4784 dict = 1.00 all = 3.5943 } Zip/UnZip{ index = 0.0957 value = 0.2875 dict = 1.00 all = 0.2782 } ---------------------------- total value len = 0.904846 GB avg = 0.119 KB (by entry num) total key len = 0.212237 GB avg = 0.028 KB total ukey len = 0.151598 GB avg = 0.020 KB total ukey num = 0.007579889 Billion total entry num = 0.007579889 Billion write speed all = 0.021530010 MB/sec (with seq num) write speed all = 0.020361287 MB/sec (without seq num) 2024/11/22-23:17:08.050997 140025003607616 [compaction_job.cc:1948] [default] [JOB 13761] Generated table #233395: 928412 keys, 37341842 bytes, temperature: kWarm 2024/11/22-23:17:08.051064 140025003607616 EVENT_LOG_v1 {"time_micros": 1732288628051030, "cf_name": "default", "job": 13761, "event": "table_file_creation", "file_number": 233395, "file_size": 37341842, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3636041955, "table_properties": {"data_size": 31112912, "index_size": 1777728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25995536, "raw_average_key_size": 28, "raw_value_size": 108222308, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928412, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288624, "oldest_key_time": 0, "file_creation_time": 1732288624, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 233395, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:17:08.086547 140025003607616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233396.sst) /dev/shm/ToplingTemp/Topling-329614-rj7Dj3 2024/11/22-23:17:09.005007 140025003607616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233396.sst): first pass time = 0.92's, 146.140'MB/sec 2024/11/22-23:17:10.319937 140025003607616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233396.sst): old prealloc_size = 147639500, real_size = 31916128 2024/11/22-23:17:10.320460 140025003607616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233396.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928021 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0299 GB avg-val = 116.63 avg-zval = 32.24 2024/11/22-23:17:10.321211 140025003607616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233396.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:17:11.215039 140025003607616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233396.sst): second pass time = 1.31's, 82.352'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 40.110'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3547.959'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.304'MB/sec write SST data time = 0.13's, 275.477'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 25314.389'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.352'MB/sec zip pipeline throughput = 82.352'MB/sec entries = 928021 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.63 avg-zval = 32.24 usrkeys = 928021 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.63 avg-zval = 32.24 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928021 bytes = 3132080 } seq expand size = 6496147 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0359 GB } UnZip/Zip{ index = 10.5151 value = 3.6176 dict = 1.00 all = 3.7385 } Zip/UnZip{ index = 0.0951 value = 0.2764 dict = 1.00 all = 0.2675 } ---------------------------- total value len = 1.019575 GB avg = 0.120 KB (by entry num) total key len = 0.238221 GB avg = 0.028 KB total ukey len = 0.170158 GB avg = 0.020 KB total ukey num = 0.008507910 Billion total entry num = 0.008507910 Billion write speed all = 0.024240556 MB/sec (with seq num) write speed all = 0.022928824 MB/sec (without seq num) 2024/11/22-23:17:11.247371 140025003607616 [compaction_job.cc:1948] [default] [JOB 13761] Generated table #233396: 928021 keys, 35901652 bytes, temperature: kWarm 2024/11/22-23:17:11.247438 140025003607616 EVENT_LOG_v1 {"time_micros": 1732288631247404, "cf_name": "default", "job": 13761, "event": "table_file_creation", "file_number": 233396, "file_size": 35901652, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3518444048, "largest_seqno": 3636042142, "table_properties": {"data_size": 29918800, "index_size": 1765120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25984588, "raw_average_key_size": 28, "raw_value_size": 108233270, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928021, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288628, "oldest_key_time": 0, "file_creation_time": 1732288628, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 233396, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:17:11.289152 140025003607616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233397.sst) /dev/shm/ToplingTemp/Topling-329614-tBGee3 2024/11/22-23:17:12.219355 140025003607616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233397.sst): first pass time = 0.93's, 144.296'MB/sec 2024/11/22-23:17:13.554947 140025003607616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233397.sst): old prealloc_size = 147639500, real_size = 33511104 2024/11/22-23:17:13.555473 140025003607616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233397.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925765 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 116.98 avg-zval = 34.05 2024/11/22-23:17:13.557463 140025003607616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233397.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:17:14.493260 140025003607616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233397.sst): second pass time = 1.33's, 81.166'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 40.202'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3509.928'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.287'MB/sec write SST data time = 0.13's, 289.628'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 30532.821'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.166'MB/sec zip pipeline throughput = 81.166'MB/sec entries = 925765 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.98 avg-zval = 34.05 usrkeys = 925765 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.98 avg-zval = 34.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925765 bytes = 3355920 } seq expand size = 6480355 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5132 value = 3.4360 dict = 1.00 all = 3.5581 } Zip/UnZip{ index = 0.0951 value = 0.2910 dict = 1.00 all = 0.2811 } ---------------------------- total value len = 1.134352 GB avg = 0.120 KB (by entry num) total key len = 0.264143 GB avg = 0.028 KB total ukey len = 0.188673 GB avg = 0.020 KB total ukey num = 0.009433675 Billion total entry num = 0.009433675 Billion write speed all = 0.026950421 MB/sec (with seq num) write speed all = 0.025496049 MB/sec (without seq num) 2024/11/22-23:17:14.526954 140025003607616 [compaction_job.cc:1948] [default] [JOB 13761] Generated table #233397: 925765 keys, 37721950 bytes, temperature: kWarm 2024/11/22-23:17:14.527018 140025003607616 EVENT_LOG_v1 {"time_micros": 1732288634526986, "cf_name": "default", "job": 13761, "event": "table_file_creation", "file_number": 233397, "file_size": 37721950, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3326330223, "largest_seqno": 3636042562, "table_properties": {"data_size": 31518320, "index_size": 1761152, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25921420, "raw_average_key_size": 28, "raw_value_size": 108296374, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925765, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288631, "oldest_key_time": 0, "file_creation_time": 1732288631, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 233397, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:17:14.565591 140025003607616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233398.sst) /dev/shm/ToplingTemp/Topling-329614-F7m7B4 2024/11/22-23:17:14.837273 140025003607616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233398.sst): first pass time = 0.27's, 242.987'MB/sec 2024/11/22-23:17:15.476745 140025003607616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233398.sst): old prealloc_size = 147639500, real_size = 17056256 2024/11/22-23:17:15.477082 140025003607616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233398.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 460788 prefix = 4 raw-key = 0.0074 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0531 GB zip-val = 0.0161 GB avg-val = 115.25 avg-zval = 34.86 2024/11/22-23:17:15.477490 140025003607616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233398.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.066 GB 2024/11/22-23:17:15.932380 140025003607616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00/233398.sst): second pass time = 0.64's, 83.134'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.46's, 41.779'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2991.598'MB/sec (index lex order gen) rebuild zvType time = 0.39's, 0.295'MB/sec write SST data time = 0.06's, 296.264'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.536'MB, throughput = 17719.682'MB/sec zip my value time = 0.64's, unzip length = 0.053'GB zip my value throughput = 83.134'MB/sec zip pipeline throughput = 83.134'MB/sec entries = 460788 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.25 avg-zval = 34.86 usrkeys = 460788 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.25 avg-zval = 34.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 460788 bytes = 1555168 } seq expand size = 3225516 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0092 GB value = 0.0531 GB dict = 0.54 MB all = 0.0660 GB } __ZipSize{ index = 0.0009 GB value = 0.0161 GB dict = 0.54 MB all = 0.0190 GB } UnZip/Zip{ index = 10.4999 value = 3.3060 dict = 1.00 all = 3.4675 } Zip/UnZip{ index = 0.0952 value = 0.3025 dict = 1.00 all = 0.2884 } ---------------------------- total value len = 1.190682 GB avg = 0.120 KB (by entry num) total key len = 0.277045 GB avg = 0.028 KB total ukey len = 0.197889 GB avg = 0.020 KB total ukey num = 0.009894463 Billion total entry num = 0.009894463 Billion write speed all = 0.028283800 MB/sec (with seq num) write speed all = 0.026758431 MB/sec (without seq num) 2024/11/22-23:17:15.952903 140025003607616 [compaction_job.cc:1948] [default] [JOB 13761] Generated table #233398: 460788 keys, 19035393 bytes, temperature: kWarm 2024/11/22-23:17:15.952967 140025003607616 EVENT_LOG_v1 {"time_micros": 1732288635952934, "cf_name": "default", "job": 13761, "event": "table_file_creation", "file_number": 233398, "file_size": 19035393, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 3636042013, "table_properties": {"data_size": 16063152, "index_size": 877696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 12902064, "raw_average_key_size": 28, "raw_value_size": 53104012, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 460788, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288634, "oldest_key_time": 0, "file_creation_time": 1732288634, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 233398, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:17:15.970445 140025003607616 [compaction_job.cc:703] job-13761: subcompact[0], size: 0.393346 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13761/att-00 [233388,233389,233390,233391,233392,233393,233394,233395,233396,233397,233398] 2024/11/22-23:17:16.043366 140025003607616 (Original Log Time 2024/11/22-23:16:42.699379) EVENT_LOG_v1 {"time_micros": 1732288602699337, "job": 13761, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [233328], "files_L3": [233384, 233385, 233266, 233267, 233268, 233223, 233224, 233225, 233226, 233227, 233228, 233357], "score": 0.997967, "input_data_size": 417937770, "oldest_snapshot_seqno": 3636588794} 2024/11/22-23:17:16.043369 140025003607616 (Original Log Time 2024/11/22-23:17:16.042714) [compaction_job.cc:2022] [default] [JOB 13761] Compacted 1@2 + 12@3 files to L3 => 393345573 bytes 2024/11/22-23:17:16.043371 140025003607616 (Original Log Time 2024/11/22-23:17:16.043300) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 11 0 0 0] max score 0.01, MB/sec: 12.6 rd, 11.8 wr, level 3, files in(1, 12) out(11 +0 blob) MB in(27.1, 371.5 +0.0 blob) out(375.1 +0.0 blob), read-write-amplify(28.5) write-amplify(13.8) OK, records in: 9957276, records dropped: 62813 output_compression: Snappy 2024/11/22-23:17:16.043379 140025003607616 (Original Log Time 2024/11/22-23:17:16.043345) EVENT_LOG_v1 {"time_micros": 1732288636043325, "cf": "default", "job": 13761, "event": "compaction_finished", "compaction_time_micros": 33271075, "compaction_time_cpu_micros": 15090014, "output_level": 3, "num_output_files": 11, "total_output_size": 393345573, "num_input_records": 9957276, "num_output_records": 9894463, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 247106689, "file_range_sync_nanos": 0, "file_fsync_nanos": 10320, "file_prepare_write_nanos": 24051, "lsm_state": [0, 0, 0, 11, 0, 0, 0]} 2024/11/22-23:17:16.185804 140025003607616 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732288602,"executesMs":33485,"compactionJobId":13761,"attempt":0,"compactionInputRawBytes":1417103605,"compactionInputZipBytes":417937770,"compactionOutputRawBytes":0,"compactionOutputZipBytes":393345573,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-ac3ea73d65c9e3e2319d61f445b37648-20e00595319e988c-00"}}, response = 2024/11/22-23:17:16.185891 140025003607616 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-13761/att-00: olev 3, work 33.485 s, result 6.457 ms, install 0.686 ms, input{raw 1.320 GiB zip 398.577 MiB}