2024/11/23-06:57:15.801733 140024886109760 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00: fork to child time = 0.009898 sec 2024/11/23-06:57:15.802509 140024886109760 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00 2024/11/23-06:57:15.810567 140024886109760 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00 2024/11/23-06:57:15.811696 140024886109760 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-06:57:15.811982 140024886109760 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-06:57:15.862922 140024886109760 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-21447/att-00/MANIFEST-206347 2024/11/23-06:57:15.876556 140024886109760 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-21447/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 249519, last_sequence is 3847890479, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 249359 2024/11/23-06:57:15.876564 140024886109760 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-06:57:15.876665 140024886109760 [version_set.cc:5721] Creating manifest 249519 2024/11/23-06:57:15.908667 140024886109760 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-06:57:15.929820 140024886109760 [compaction_job.cc:2437] [default] [JOB 21447] Compacting 1@2 + 11@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-06:57:15.929845 140024886109760 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 16068 Base level 2, inputs: [249514(19MB)], [249443(34MB) 249444(33MB) 249445(36MB) 249446(33MB) 249447(34MB) 249448(35MB) 249449(35MB) 249450(35MB) 249451(35MB) 249452(35MB) 249453(19MB)] 2024/11/23-06:57:15.935190 140024886109760 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-06:57:15.935227 140024886109760 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-06:57:15.935248 140024886109760 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-06:57:15.935253 140024886109760 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-06:57:15.935263 140024886109760 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-06:57:16.361447 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249520.sst) /dev/shm/ToplingTemp/Topling-440941-DpyZv6 2024/11/23-06:57:17.248589 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249520.sst): first pass time = 0.89's, 151.300'MB/sec 2024/11/23-06:57:20.280547 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249520.sst): old prealloc_size = 147639500, real_size = 32681792 2024/11/23-06:57:20.281156 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249520.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930839 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0307 GB avg-val = 116.19 avg-zval = 32.95 2024/11/23-06:57:20.281905 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249520.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:57:21.778920 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249520.sst): second pass time = 3.03's, 35.681'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.50's, 24.352'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3015.583'MB/sec (index lex order gen) rebuild zvType time = 1.21's, 0.192'MB/sec write SST data time = 0.29's, 126.335'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 28921.637'MB/sec zip my value time = 3.03's, unzip length = 0.108'GB zip my value throughput = 35.681'MB/sec zip pipeline throughput = 135.693'MB/sec entries = 930839 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.19 avg-zval = 32.95 usrkeys = 930839 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.81 avg-zval = 32.95 TagRS{ kinds = RS_Key0_TagN bytes = 145560 } TagArray{ size = 747842 bytes = 2804416 } seq expand size = 5234894 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.4467 value = 3.5268 dict = 1.00 all = 3.6785 } Zip/UnZip{ index = 0.0957 value = 0.2835 dict = 1.00 all = 0.2718 } ---------------------------- total value len = 0.113389 GB avg = 0.122 KB (by entry num) total key len = 0.026063 GB avg = 0.028 KB total ukey len = 0.018617 GB avg = 0.020 KB total ukey num = 0.000930839 Billion total entry num = 0.000930839 Billion write speed all = 0.001754152 MB/sec (with seq num) write speed all = 0.001660481 MB/sec (without seq num) 2024/11/23-06:57:21.813960 140024886109760 [compaction_job.cc:1948] [default] [JOB 21447] Generated table #249520: 930839 keys, 36487086 bytes, temperature: kWarm 2024/11/23-06:57:21.814048 140024886109760 EVENT_LOG_v1 {"time_micros": 1732316241813997, "cf_name": "default", "job": 21447, "event": "table_file_creation", "file_number": 249520, "file_size": 36487086, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3847864884, "table_properties": {"data_size": 30666800, "index_size": 1782080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26063492, "raw_average_key_size": 28, "raw_value_size": 108154354, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930839, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732316236, "oldest_key_time": 0, "file_creation_time": 1732316236, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249520, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:57:21.854503 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249521.sst) /dev/shm/ToplingTemp/Topling-440941-4uuTv7 2024/11/23-06:57:23.402851 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249521.sst): first pass time = 1.55's, 86.686'MB/sec 2024/11/23-06:57:25.344423 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249521.sst): old prealloc_size = 147639500, real_size = 30910992 2024/11/23-06:57:25.344998 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249521.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924709 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0289 GB avg-val = 117.15 avg-zval = 31.25 2024/11/23-06:57:25.345776 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249521.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:57:26.189937 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249521.sst): second pass time = 1.94's, 55.838'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 41.672'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3197.319'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 271.400'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35770.302'MB/sec zip my value time = 1.94's, unzip length = 0.108'GB zip my value throughput = 55.838'MB/sec zip pipeline throughput = 101.628'MB/sec entries = 924709 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.15 avg-zval = 31.25 usrkeys = 924709 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.15 avg-zval = 31.25 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924709 bytes = 3467680 } seq expand size = 6472963 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0352 GB } UnZip/Zip{ index = 10.3775 value = 3.7486 dict = 1.00 all = 3.8095 } Zip/UnZip{ index = 0.0964 value = 0.2668 dict = 1.00 all = 0.2625 } ---------------------------- total value len = 0.228188 GB avg = 0.123 KB (by entry num) total key len = 0.051955 GB avg = 0.028 KB total ukey len = 0.037111 GB avg = 0.020 KB total ukey num = 0.001855548 Billion total entry num = 0.001855548 Billion write speed all = 0.003523680 MB/sec (with seq num) write speed all = 0.003336965 MB/sec (without seq num) 2024/11/23-06:57:26.227296 140024886109760 [compaction_job.cc:1948] [default] [JOB 21447] Generated table #249521: 924709 keys, 35232812 bytes, temperature: kWarm 2024/11/23-06:57:26.227357 140024886109760 EVENT_LOG_v1 {"time_micros": 1732316246227326, "cf_name": "default", "job": 21447, "event": "table_file_creation", "file_number": 249521, "file_size": 35232812, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3279302997, "largest_seqno": 3847861138, "table_properties": {"data_size": 28897472, "index_size": 1782144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25891852, "raw_average_key_size": 28, "raw_value_size": 108325898, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924709, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732316241, "oldest_key_time": 0, "file_creation_time": 1732316241, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249521, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:57:26.265300 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249522.sst) /dev/shm/ToplingTemp/Topling-440941-KYQvd7 2024/11/23-06:57:27.275006 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249522.sst): first pass time = 1.01's, 132.933'MB/sec 2024/11/23-06:57:29.410774 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249522.sst): old prealloc_size = 147639500, real_size = 33465552 2024/11/23-06:57:29.411420 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249522.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933067 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0315 GB avg-val = 115.85 avg-zval = 33.71 2024/11/23-06:57:29.412265 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249522.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:57:30.671816 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249522.sst): second pass time = 2.13's, 50.650'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.26's, 29.895'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2924.934'MB/sec (index lex order gen) rebuild zvType time = 1.09's, 0.215'MB/sec write SST data time = 0.17's, 216.731'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33595.399'MB/sec zip my value time = 2.13's, unzip length = 0.108'GB zip my value throughput = 50.650'MB/sec zip pipeline throughput = 55.957'MB/sec entries = 933067 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.85 avg-zval = 33.71 usrkeys = 933067 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.85 avg-zval = 33.71 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933067 bytes = 3382384 } seq expand size = 6531469 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5136 value = 3.4362 dict = 1.00 all = 3.5603 } Zip/UnZip{ index = 0.0951 value = 0.2910 dict = 1.00 all = 0.2809 } ---------------------------- total value len = 0.342812 GB avg = 0.123 KB (by entry num) total key len = 0.078081 GB avg = 0.028 KB total ukey len = 0.055772 GB avg = 0.020 KB total ukey num = 0.002788615 Billion total entry num = 0.002788615 Billion write speed all = 0.005293744 MB/sec (with seq num) write speed all = 0.005013156 MB/sec (without seq num) 2024/11/23-06:57:30.706034 140024886109760 [compaction_job.cc:1948] [default] [JOB 21447] Generated table #249522: 933067 keys, 37698334 bytes, temperature: kWarm 2024/11/23-06:57:30.706104 140024886109760 EVENT_LOG_v1 {"time_micros": 1732316250706066, "cf_name": "default", "job": 21447, "event": "table_file_creation", "file_number": 249522, "file_size": 37698334, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 3847866035, "table_properties": {"data_size": 31457152, "index_size": 1774976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26125876, "raw_average_key_size": 28, "raw_value_size": 108091966, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933067, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732316246, "oldest_key_time": 0, "file_creation_time": 1732316246, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249522, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:57:30.745321 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249523.sst) /dev/shm/ToplingTemp/Topling-440941-0yeOg5 2024/11/23-06:57:31.793919 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249523.sst): first pass time = 1.05's, 128.003'MB/sec 2024/11/23-06:57:33.512954 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249523.sst): old prealloc_size = 147639500, real_size = 30784512 2024/11/23-06:57:33.513641 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249523.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928645 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0288 GB avg-val = 116.53 avg-zval = 31.00 2024/11/23-06:57:33.514477 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249523.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:57:34.545220 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249523.sst): second pass time = 1.72's, 62.981'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 33.798'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2657.956'MB/sec (index lex order gen) rebuild zvType time = 0.88's, 0.265'MB/sec write SST data time = 0.15's, 225.992'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36262.744'MB/sec zip my value time = 1.72's, unzip length = 0.108'GB zip my value throughput = 62.981'MB/sec zip pipeline throughput = 62.981'MB/sec entries = 928645 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.53 avg-zval = 31.00 usrkeys = 928645 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.53 avg-zval = 31.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928645 bytes = 3250272 } seq expand size = 6500515 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0288 GB dict = 1.08 MB all = 0.0349 GB } UnZip/Zip{ index = 10.5134 value = 3.7594 dict = 1.00 all = 3.8471 } Zip/UnZip{ index = 0.0951 value = 0.2660 dict = 1.00 all = 0.2599 } ---------------------------- total value len = 0.457528 GB avg = 0.123 KB (by entry num) total key len = 0.104083 GB avg = 0.028 KB total ukey len = 0.074345 GB avg = 0.020 KB total ukey num = 0.003717260 Billion total entry num = 0.003717260 Billion write speed all = 0.007063274 MB/sec (with seq num) write speed all = 0.006689264 MB/sec (without seq num) 2024/11/23-06:57:34.577348 140024886109760 [compaction_job.cc:1948] [default] [JOB 21447] Generated table #249523: 928645 keys, 34888296 bytes, temperature: kWarm 2024/11/23-06:57:34.577411 140024886109760 EVENT_LOG_v1 {"time_micros": 1732316254577380, "cf_name": "default", "job": 21447, "event": "table_file_creation", "file_number": 249523, "file_size": 34888296, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3604582272, "largest_seqno": 3847865847, "table_properties": {"data_size": 28785584, "index_size": 1766592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26002060, "raw_average_key_size": 28, "raw_value_size": 108215806, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928645, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732316250, "oldest_key_time": 0, "file_creation_time": 1732316250, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249523, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:57:34.613920 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249524.sst) /dev/shm/ToplingTemp/Topling-440941-oB3tB5 2024/11/23-06:57:35.668505 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249524.sst): first pass time = 1.05's, 127.275'MB/sec 2024/11/23-06:57:37.507693 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249524.sst): old prealloc_size = 147639500, real_size = 32020432 2024/11/23-06:57:37.508298 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249524.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928741 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0300 GB avg-val = 116.52 avg-zval = 32.30 2024/11/23-06:57:37.509160 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249524.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:57:38.512717 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249524.sst): second pass time = 1.84's, 58.885'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.00's, 36.060'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3049.893'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.300'MB/sec write SST data time = 0.23's, 156.565'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32472.258'MB/sec zip my value time = 1.84's, unzip length = 0.108'GB zip my value throughput = 58.885'MB/sec zip pipeline throughput = 88.753'MB/sec entries = 928741 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.52 avg-zval = 32.30 usrkeys = 928741 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.52 avg-zval = 32.30 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928741 bytes = 3366704 } seq expand size = 6501187 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.3750 value = 3.6074 dict = 1.00 all = 3.7035 } Zip/UnZip{ index = 0.0964 value = 0.2772 dict = 1.00 all = 0.2700 } ---------------------------- total value len = 0.572242 GB avg = 0.123 KB (by entry num) total key len = 0.130088 GB avg = 0.028 KB total ukey len = 0.092920 GB avg = 0.020 KB total ukey num = 0.004646001 Billion total entry num = 0.004646001 Billion write speed all = 0.008832628 MB/sec (with seq num) write speed all = 0.008365196 MB/sec (without seq num) 2024/11/23-06:57:38.547630 140024886109760 [compaction_job.cc:1948] [default] [JOB 21447] Generated table #249524: 928741 keys, 36240520 bytes, temperature: kWarm 2024/11/23-06:57:38.547711 140024886109760 EVENT_LOG_v1 {"time_micros": 1732316258547668, "cf_name": "default", "job": 21447, "event": "table_file_creation", "file_number": 249524, "file_size": 36240520, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3485447251, "largest_seqno": 3847866132, "table_properties": {"data_size": 29997696, "index_size": 1790336, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26004748, "raw_average_key_size": 28, "raw_value_size": 108212990, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928741, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732316254, "oldest_key_time": 0, "file_creation_time": 1732316254, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249524, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:57:38.613967 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249525.sst) /dev/shm/ToplingTemp/Topling-440941-bcDqW4 2024/11/23-06:57:39.592419 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249525.sst): first pass time = 0.98's, 137.181'MB/sec 2024/11/23-06:57:41.221899 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249525.sst): old prealloc_size = 147639500, real_size = 32397760 2024/11/23-06:57:41.222433 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249525.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928725 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0304 GB avg-val = 116.52 avg-zval = 32.72 2024/11/23-06:57:41.223221 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249525.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:57:42.133476 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249525.sst): second pass time = 1.63's, 66.455'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 40.170'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3458.578'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.298'MB/sec write SST data time = 0.13's, 276.315'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 37951.079'MB/sec zip my value time = 1.63's, unzip length = 0.108'GB zip my value throughput = 66.455'MB/sec zip pipeline throughput = 66.455'MB/sec entries = 928725 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.52 avg-zval = 32.72 usrkeys = 928725 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.52 avg-zval = 32.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928725 bytes = 3366640 } seq expand size = 6501075 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.4436 value = 3.5612 dict = 1.00 all = 3.6654 } Zip/UnZip{ index = 0.0958 value = 0.2808 dict = 1.00 all = 0.2728 } ---------------------------- total value len = 0.686957 GB avg = 0.123 KB (by entry num) total key len = 0.156092 GB avg = 0.028 KB total ukey len = 0.111495 GB avg = 0.020 KB total ukey num = 0.005574726 Billion total entry num = 0.005574726 Billion write speed all = 0.010601851 MB/sec (with seq num) write speed all = 0.010041006 MB/sec (without seq num) 2024/11/23-06:57:42.168081 140024886109760 [compaction_job.cc:1948] [default] [JOB 21447] Generated table #249525: 928725 keys, 36617860 bytes, temperature: kWarm 2024/11/23-06:57:42.168142 140024886109760 EVENT_LOG_v1 {"time_micros": 1732316262168112, "cf_name": "default", "job": 21447, "event": "table_file_creation", "file_number": 249525, "file_size": 36617860, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 3847864272, "table_properties": {"data_size": 30386864, "index_size": 1778560, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26004300, "raw_average_key_size": 28, "raw_value_size": 108213518, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928725, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732316258, "oldest_key_time": 0, "file_creation_time": 1732316258, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249525, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:57:42.203991 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249526.sst) /dev/shm/ToplingTemp/Topling-440941-2p2VC7 2024/11/23-06:57:43.048247 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249526.sst): first pass time = 0.84's, 158.985'MB/sec 2024/11/23-06:57:44.526330 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249526.sst): old prealloc_size = 147639500, real_size = 33304704 2024/11/23-06:57:44.526988 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249526.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928659 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0313 GB avg-val = 116.53 avg-zval = 33.69 2024/11/23-06:57:44.527836 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249526.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:57:45.500331 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249526.sst): second pass time = 1.48's, 73.278'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.97's, 38.527'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2821.442'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.279'MB/sec write SST data time = 0.14's, 267.387'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30836.646'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.278'MB/sec zip pipeline throughput = 73.278'MB/sec entries = 928659 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.53 avg-zval = 33.69 usrkeys = 928659 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.53 avg-zval = 33.69 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928659 bytes = 3366400 } seq expand size = 6500613 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.3749 value = 3.4593 dict = 1.00 all = 3.5768 } Zip/UnZip{ index = 0.0964 value = 0.2891 dict = 1.00 all = 0.2796 } ---------------------------- total value len = 0.801673 GB avg = 0.123 KB (by entry num) total key len = 0.182095 GB avg = 0.028 KB total ukey len = 0.130068 GB avg = 0.020 KB total ukey num = 0.006503385 Billion total entry num = 0.006503385 Billion write speed all = 0.012370945 MB/sec (with seq num) write speed all = 0.011716701 MB/sec (without seq num) 2024/11/23-06:57:45.532349 140024886109760 [compaction_job.cc:1948] [default] [JOB 21447] Generated table #249526: 928659 keys, 37524592 bytes, temperature: kWarm 2024/11/23-06:57:45.532409 140024886109760 EVENT_LOG_v1 {"time_micros": 1732316265532379, "cf_name": "default", "job": 21447, "event": "table_file_creation", "file_number": 249526, "file_size": 37524592, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 3847865250, "table_properties": {"data_size": 31282160, "index_size": 1790208, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26002452, "raw_average_key_size": 28, "raw_value_size": 108215298, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928659, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732316262, "oldest_key_time": 0, "file_creation_time": 1732316262, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249526, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:57:45.566633 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249527.sst) /dev/shm/ToplingTemp/Topling-440941-sID3t3 2024/11/23-06:57:46.416962 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249527.sst): first pass time = 0.85's, 157.851'MB/sec 2024/11/23-06:57:47.883580 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249527.sst): old prealloc_size = 147639500, real_size = 33084176 2024/11/23-06:57:47.884217 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249527.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928727 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.52 avg-zval = 33.47 2024/11/23-06:57:47.884990 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249527.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:57:48.828963 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249527.sst): second pass time = 1.47's, 73.819'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.337'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2870.869'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.285'MB/sec write SST data time = 0.13's, 287.619'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33615.715'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.819'MB/sec zip pipeline throughput = 73.819'MB/sec entries = 928727 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.52 avg-zval = 33.47 usrkeys = 928727 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.52 avg-zval = 33.47 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928727 bytes = 3250560 } seq expand size = 6501089 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5117 value = 3.4812 dict = 1.00 all = 3.6091 } Zip/UnZip{ index = 0.0951 value = 0.2873 dict = 1.00 all = 0.2771 } ---------------------------- total value len = 0.916387 GB avg = 0.123 KB (by entry num) total key len = 0.208099 GB avg = 0.028 KB total ukey len = 0.148642 GB avg = 0.020 KB total ukey num = 0.007432112 Billion total entry num = 0.007432112 Billion write speed all = 0.014139904 MB/sec (with seq num) write speed all = 0.013392260 MB/sec (without seq num) 2024/11/23-06:57:48.863872 140024886109760 [compaction_job.cc:1948] [default] [JOB 21447] Generated table #249527: 928727 keys, 37188236 bytes, temperature: kWarm 2024/11/23-06:57:48.863935 140024886109760 EVENT_LOG_v1 {"time_micros": 1732316268863905, "cf_name": "default", "job": 21447, "event": "table_file_creation", "file_number": 249527, "file_size": 37188236, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3596991077, "largest_seqno": 3847865316, "table_properties": {"data_size": 31084800, "index_size": 1767040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26004356, "raw_average_key_size": 28, "raw_value_size": 108213418, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928727, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732316265, "oldest_key_time": 0, "file_creation_time": 1732316265, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249527, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:57:48.903367 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249528.sst) /dev/shm/ToplingTemp/Topling-440941-Ee79s4 2024/11/23-06:57:49.788028 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249528.sst): first pass time = 0.88's, 151.723'MB/sec 2024/11/23-06:57:51.282352 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249528.sst): old prealloc_size = 147639500, real_size = 33046896 2024/11/23-06:57:51.282920 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249528.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928899 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.49 avg-zval = 33.42 2024/11/23-06:57:51.284726 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249528.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:57:52.223462 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249528.sst): second pass time = 1.49's, 72.475'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.600'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3219.556'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.287'MB/sec write SST data time = 0.13's, 285.055'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31637.967'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.475'MB/sec zip pipeline throughput = 72.475'MB/sec entries = 928899 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.49 avg-zval = 33.42 usrkeys = 928899 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.49 avg-zval = 33.42 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928899 bytes = 3367280 } seq expand size = 6502293 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5110 value = 3.4853 dict = 1.00 all = 3.6015 } Zip/UnZip{ index = 0.0951 value = 0.2869 dict = 1.00 all = 0.2777 } ---------------------------- total value len = 1.031098 GB avg = 0.123 KB (by entry num) total key len = 0.234108 GB avg = 0.028 KB total ukey len = 0.167220 GB avg = 0.020 KB total ukey num = 0.008361011 Billion total entry num = 0.008361011 Billion write speed all = 0.015908716 MB/sec (with seq num) write speed all = 0.015067665 MB/sec (without seq num) 2024/11/23-06:57:52.257108 140024886109760 [compaction_job.cc:1948] [default] [JOB 21447] Generated table #249528: 928899 keys, 37267406 bytes, temperature: kWarm 2024/11/23-06:57:52.257171 140024886109760 EVENT_LOG_v1 {"time_micros": 1732316272257137, "cf_name": "default", "job": 21447, "event": "table_file_creation", "file_number": 249528, "file_size": 37267406, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 3847863374, "table_properties": {"data_size": 31047008, "index_size": 1767488, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26009172, "raw_average_key_size": 28, "raw_value_size": 108208654, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928899, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732316268, "oldest_key_time": 0, "file_creation_time": 1732316268, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249528, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:57:52.293216 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249529.sst) /dev/shm/ToplingTemp/Topling-440941-N7MjZ3 2024/11/23-06:57:53.111881 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249529.sst): first pass time = 0.82's, 163.955'MB/sec 2024/11/23-06:57:54.754377 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249529.sst): old prealloc_size = 147639500, real_size = 33306672 2024/11/23-06:57:54.755005 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249529.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928485 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0313 GB avg-val = 116.56 avg-zval = 33.71 2024/11/23-06:57:54.756933 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249529.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:57:55.725680 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249529.sst): second pass time = 1.64's, 65.938'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.97's, 38.635'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2982.652'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.282'MB/sec write SST data time = 0.15's, 254.844'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35668.759'MB/sec zip my value time = 1.64's, unzip length = 0.108'GB zip my value throughput = 65.938'MB/sec zip pipeline throughput = 108.959'MB/sec entries = 928485 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.56 avg-zval = 33.71 usrkeys = 928485 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.56 avg-zval = 33.71 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928485 bytes = 3365776 } seq expand size = 6499395 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4454 value = 3.4579 dict = 1.00 all = 3.5767 } Zip/UnZip{ index = 0.0957 value = 0.2892 dict = 1.00 all = 0.2796 } ---------------------------- total value len = 1.145818 GB avg = 0.123 KB (by entry num) total key len = 0.260106 GB avg = 0.028 KB total ukey len = 0.185790 GB avg = 0.020 KB total ukey num = 0.009289496 Billion total entry num = 0.009289496 Billion write speed all = 0.017677316 MB/sec (with seq num) write speed all = 0.016742908 MB/sec (without seq num) 2024/11/23-06:57:55.758649 140024886109760 [compaction_job.cc:1948] [default] [JOB 21447] Generated table #249529: 928485 keys, 37525932 bytes, temperature: kWarm 2024/11/23-06:57:55.758709 140024886109760 EVENT_LOG_v1 {"time_micros": 1732316275758679, "cf_name": "default", "job": 21447, "event": "table_file_creation", "file_number": 249529, "file_size": 37525932, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 3847865681, "table_properties": {"data_size": 31296544, "index_size": 1777792, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25997580, "raw_average_key_size": 28, "raw_value_size": 108220182, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928485, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732316272, "oldest_key_time": 0, "file_creation_time": 1732316272, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249529, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:57:55.793292 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249530.sst) /dev/shm/ToplingTemp/Topling-440941-h1VBy5 2024/11/23-06:57:56.299855 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249530.sst): first pass time = 0.51's, 178.118'MB/sec 2024/11/23-06:57:57.246768 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249530.sst): old prealloc_size = 147639500, real_size = 21472656 2024/11/23-06:57:57.247253 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249530.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 627429 prefix = 4 raw-key = 0.0100 GB zip-key = 0.0012 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0727 GB zip-val = 0.0201 GB avg-val = 115.80 avg-zval = 32.06 2024/11/23-06:57:57.247828 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249530.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0016 GB, waited 0.000 sec, Key+Value = 0.090 GB 2024/11/23-06:57:57.980881 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00/249530.sst): second pass time = 0.95's, 76.789'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.73's, 32.812'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2656.125'MB/sec (index lex order gen) rebuild zvType time = 0.63's, 0.250'MB/sec write SST data time = 0.11's, 228.529'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.729'MB, throughput = 23125.539'MB/sec zip my value time = 0.95's, unzip length = 0.073'GB zip my value throughput = 76.789'MB/sec zip pipeline throughput = 76.789'MB/sec entries = 627429 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.80 avg-zval = 32.06 usrkeys = 627429 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.80 avg-zval = 32.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 627429 bytes = 2039152 } seq expand size = 4392003 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0125 GB value = 0.0727 GB dict = 0.73 MB all = 0.0902 GB } __ZipSize{ index = 0.0012 GB value = 0.0201 GB dict = 0.73 MB all = 0.0241 GB } UnZip/Zip{ index = 10.4393 value = 3.6122 dict = 1.00 all = 3.7456 } Zip/UnZip{ index = 0.0958 value = 0.2768 dict = 1.00 all = 0.2670 } ---------------------------- total value len = 1.222863 GB avg = 0.123 KB (by entry num) total key len = 0.277674 GB avg = 0.028 KB total ukey len = 0.198339 GB avg = 0.020 KB total ukey num = 0.009916925 Billion total entry num = 0.009916925 Billion write speed all = 0.018866396 MB/sec (with seq num) write speed all = 0.017868905 MB/sec (without seq num) 2024/11/23-06:57:58.004846 140024886109760 [compaction_job.cc:1948] [default] [JOB 21447] Generated table #249530: 627429 keys, 24087069 bytes, temperature: kWarm 2024/11/23-06:57:58.004911 140024886109760 EVENT_LOG_v1 {"time_micros": 1732316278004877, "cf_name": "default", "job": 21447, "event": "table_file_creation", "file_number": 249530, "file_size": 24087069, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3788029349, "largest_seqno": 3847863931, "table_properties": {"data_size": 20113536, "index_size": 1202048, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 17568012, "raw_average_key_size": 28, "raw_value_size": 72653186, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 627429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732316275, "oldest_key_time": 0, "file_creation_time": 1732316275, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249530, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:57:58.068623 140024886109760 [compaction_job.cc:703] job-21447: subcompact[0], size: 0.390758 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21447/att-00 [249520,249521,249522,249523,249524,249525,249526,249527,249528,249529,249530] 2024/11/23-06:57:58.141156 140024886109760 (Original Log Time 2024/11/23-06:57:15.929912) EVENT_LOG_v1 {"time_micros": 1732316235929862, "job": 21447, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [249514], "files_L3": [249443, 249444, 249445, 249446, 249447, 249448, 249449, 249450, 249451, 249452, 249453], "score": 0.998315, "input_data_size": 409456871, "oldest_snapshot_seqno": 3847890396} 2024/11/23-06:57:58.141160 140024886109760 (Original Log Time 2024/11/23-06:57:58.140528) [compaction_job.cc:2022] [default] [JOB 21447] Compacted 1@2 + 11@3 files to L3 => 390758143 bytes 2024/11/23-06:57:58.141161 140024886109760 (Original Log Time 2024/11/23-06:57:58.141094) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 11 0 0 0] max score 0.01, MB/sec: 9.7 rd, 9.3 wr, level 3, files in(1, 11) out(11 +0 blob) MB in(19.9, 370.6 +0.0 blob) out(372.7 +0.0 blob), read-write-amplify(38.4) write-amplify(18.7) OK, records in: 9992651, records dropped: 75726 output_compression: Snappy 2024/11/23-06:57:58.141167 140024886109760 (Original Log Time 2024/11/23-06:57:58.141136) EVENT_LOG_v1 {"time_micros": 1732316278141117, "cf": "default", "job": 21447, "event": "compaction_finished", "compaction_time_micros": 42138719, "compaction_time_cpu_micros": 17887740, "output_level": 3, "num_output_files": 11, "total_output_size": 390758143, "num_input_records": 9992651, "num_output_records": 9916925, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 272665237, "file_range_sync_nanos": 0, "file_fsync_nanos": 9482, "file_prepare_write_nanos": 26059, "lsm_state": [0, 0, 0, 11, 0, 0, 0]} 2024/11/23-06:57:58.272364 140024886109760 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732316235,"executesMs":42332,"compactionJobId":21447,"attempt":0,"compactionInputRawBytes":1443152262,"compactionInputZipBytes":409456871,"compactionOutputRawBytes":0,"compactionOutputZipBytes":390758143,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-76d9f379cb8917db05336d710253d2a5-2d47edf0ace1f9c4-00"}}, response = 2024/11/23-06:57:58.272433 140024886109760 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-21447/att-00: olev 3, work 42.332 s, result 6.273 ms, install 0.657 ms, input{raw 1.344 GiB zip 390.489 MiB}