2024/11/23-13:37:15.233488 139957433374272 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00: fork to child time = 0.007166 sec 2024/11/23-13:37:15.234170 139957433374272 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00 2024/11/23-13:37:15.240617 139957433374272 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00 2024/11/23-13:37:15.241479 139957433374272 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-13:37:15.241684 139957433374272 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-13:37:15.296064 139957433374272 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-26939/att-00/MANIFEST-206347 2024/11/23-13:37:15.338726 139957433374272 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-26939/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 261498, last_sequence is 3999400876, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 261290 2024/11/23-13:37:15.338733 139957433374272 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-13:37:15.338813 139957433374272 [version_set.cc:5721] Creating manifest 261498 2024/11/23-13:37:15.372027 139957433374272 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-13:37:15.394436 139957433374272 [compaction_job.cc:2437] [default] [JOB 26939] Compacting 1@2 + 11@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-13:37:15.394446 139957433374272 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 20175 Base level 2, inputs: [261426(27MB)], [261376(34MB) 261377(35MB) 261378(35MB) 261379(36MB) 261380(36MB) 261381(37MB) 261382(35MB) 261383(34MB) 261384(36MB) 261385(35MB) 261482(34MB)] 2024/11/23-13:37:15.403038 139957433374272 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-13:37:15.403066 139957433374272 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-13:37:15.403080 139957433374272 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-13:37:15.403084 139957433374272 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-13:37:15.403094 139957433374272 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-13:37:15.932763 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261499.sst) /dev/shm/ToplingTemp/Topling-61810-kzuiDX 2024/11/23-13:37:16.530640 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261499.sst): first pass time = 0.60's, 224.509'MB/sec 2024/11/23-13:37:18.432743 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261499.sst): old prealloc_size = 147639500, real_size = 31878528 2024/11/23-13:37:18.433422 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261499.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930479 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0299 GB avg-val = 116.25 avg-zval = 32.11 2024/11/23-13:37:18.434338 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261499.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:37:19.597008 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261499.sst): second pass time = 1.90's, 56.911'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.16's, 30.810'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2689.116'MB/sec (index lex order gen) rebuild zvType time = 1.01's, 0.231'MB/sec write SST data time = 0.16's, 228.508'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 29861.723'MB/sec zip my value time = 1.90's, unzip length = 0.108'GB zip my value throughput = 56.911'MB/sec zip pipeline throughput = 56.911'MB/sec entries = 930479 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.25 avg-zval = 32.11 usrkeys = 930479 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.25 avg-zval = 32.11 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930479 bytes = 3140384 } seq expand size = 6513353 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0359 GB } UnZip/Zip{ index = 10.5125 value = 3.6205 dict = 1.00 all = 3.7417 } Zip/UnZip{ index = 0.0951 value = 0.2762 dict = 1.00 all = 0.2673 } ---------------------------- total value len = 8.963297 GB avg = 0.168 KB (by entry num) total key len = 1.466874 GB avg = 0.028 KB total ukey len = 1.016488 GB avg = 0.015 KB total ukey num = 0.066480914 Billion total entry num = 0.053288643 Billion write speed all = 0.616171583 MB/sec (with seq num) write speed all = 0.590986996 MB/sec (without seq num) 2024/11/23-13:37:19.642727 139957433374272 [compaction_job.cc:1948] [default] [JOB 26939] Generated table #261499: 930479 keys, 35870630 bytes, temperature: kWarm 2024/11/23-13:37:19.642820 139957433374272 EVENT_LOG_v1 {"time_micros": 1732340239642763, "cf_name": "default", "job": 26939, "event": "table_file_creation", "file_number": 261499, "file_size": 35870630, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3925603882, "largest_seqno": 3999056765, "table_properties": {"data_size": 29875504, "index_size": 1770240, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26053412, "raw_average_key_size": 28, "raw_value_size": 108164346, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930479, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340235, "oldest_key_time": 0, "file_creation_time": 1732340235, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261499, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:37:19.679411 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261500.sst) /dev/shm/ToplingTemp/Topling-61810-gHf0uZ 2024/11/23-13:37:20.651175 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261500.sst): first pass time = 0.97's, 138.123'MB/sec 2024/11/23-13:37:22.402736 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261500.sst): old prealloc_size = 147639500, real_size = 32646080 2024/11/23-13:37:22.403325 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261500.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930697 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0306 GB avg-val = 116.21 avg-zval = 32.92 2024/11/23-13:37:22.404120 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261500.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:37:23.264772 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261500.sst): second pass time = 1.75's, 61.785'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.774'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3124.417'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 286.789'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 28267.593'MB/sec zip my value time = 1.75's, unzip length = 0.108'GB zip my value throughput = 61.785'MB/sec zip pipeline throughput = 91.748'MB/sec entries = 930697 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.21 avg-zval = 32.92 usrkeys = 930697 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.21 avg-zval = 32.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930697 bytes = 3373792 } seq expand size = 6514879 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5130 value = 3.5297 dict = 1.00 all = 3.6401 } Zip/UnZip{ index = 0.0951 value = 0.2833 dict = 1.00 all = 0.2747 } ---------------------------- total value len = 9.077971 GB avg = 0.167 KB (by entry num) total key len = 1.492934 GB avg = 0.028 KB total ukey len = 1.035102 GB avg = 0.015 KB total ukey num = 0.067411611 Billion total entry num = 0.054219340 Billion write speed all = 0.624350208 MB/sec (with seq num) write speed all = 0.598731317 MB/sec (without seq num) 2024/11/23-13:37:23.313265 139957433374272 [compaction_job.cc:1948] [default] [JOB 26939] Generated table #261500: 930697 keys, 36871590 bytes, temperature: kWarm 2024/11/23-13:37:23.313330 139957433374272 EVENT_LOG_v1 {"time_micros": 1732340243313297, "cf_name": "default", "job": 26939, "event": "table_file_creation", "file_number": 261500, "file_size": 36871590, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3711387651, "largest_seqno": 3999056775, "table_properties": {"data_size": 30642672, "index_size": 1770560, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26059516, "raw_average_key_size": 28, "raw_value_size": 108158290, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930697, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340239, "oldest_key_time": 0, "file_creation_time": 1732340239, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261500, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:37:23.348224 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261501.sst) /dev/shm/ToplingTemp/Topling-61810-5J2JXX 2024/11/23-13:37:24.231511 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261501.sst): first pass time = 0.88's, 151.959'MB/sec 2024/11/23-13:37:25.771633 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261501.sst): old prealloc_size = 147639500, real_size = 32856560 2024/11/23-13:37:25.772196 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261501.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930850 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.19 avg-zval = 33.14 2024/11/23-13:37:25.772982 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261501.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:37:26.643853 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261501.sst): second pass time = 1.54's, 70.258'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.383'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3248.037'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.13's, 279.337'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 37523.521'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.258'MB/sec zip pipeline throughput = 105.266'MB/sec entries = 930850 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.19 avg-zval = 33.14 usrkeys = 930850 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.19 avg-zval = 33.14 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930850 bytes = 3257984 } seq expand size = 6515950 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5110 value = 3.5055 dict = 1.00 all = 3.6308 } Zip/UnZip{ index = 0.0951 value = 0.2853 dict = 1.00 all = 0.2754 } ---------------------------- total value len = 9.192641 GB avg = 0.167 KB (by entry num) total key len = 1.518998 GB avg = 0.028 KB total ukey len = 1.053719 GB avg = 0.015 KB total ukey num = 0.068342461 Billion total entry num = 0.055150190 Billion write speed all = 0.632536137 MB/sec (with seq num) write speed all = 0.606482615 MB/sec (without seq num) 2024/11/23-13:37:26.688471 139957433374272 [compaction_job.cc:1948] [default] [JOB 26939] Generated table #261501: 930850 keys, 36966122 bytes, temperature: kWarm 2024/11/23-13:37:26.688536 139957433374272 EVENT_LOG_v1 {"time_micros": 1732340246688504, "cf_name": "default", "job": 26939, "event": "table_file_creation", "file_number": 261501, "file_size": 36966122, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3855348712, "largest_seqno": 3999056790, "table_properties": {"data_size": 30852448, "index_size": 1771200, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26063800, "raw_average_key_size": 28, "raw_value_size": 108154016, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930850, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340243, "oldest_key_time": 0, "file_creation_time": 1732340243, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261501, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:37:26.724757 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261502.sst) /dev/shm/ToplingTemp/Topling-61810-DsfH2X 2024/11/23-13:37:27.644402 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261502.sst): first pass time = 0.92's, 145.950'MB/sec 2024/11/23-13:37:29.102999 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261502.sst): old prealloc_size = 147639500, real_size = 33713040 2024/11/23-13:37:29.103557 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261502.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930526 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0317 GB avg-val = 116.24 avg-zval = 34.06 2024/11/23-13:37:29.104348 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261502.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:37:29.960237 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261502.sst): second pass time = 1.46's, 74.196'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.985'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3255.107'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.13's, 290.330'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35994.060'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 74.196'MB/sec zip pipeline throughput = 74.196'MB/sec entries = 930526 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.24 avg-zval = 34.06 usrkeys = 930526 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.24 avg-zval = 34.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930526 bytes = 3140544 } seq expand size = 6513682 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.4443 value = 3.4123 dict = 1.00 all = 3.5596 } Zip/UnZip{ index = 0.0957 value = 0.2931 dict = 1.00 all = 0.2809 } ---------------------------- total value len = 9.307317 GB avg = 0.166 KB (by entry num) total key len = 1.545053 GB avg = 0.028 KB total ukey len = 1.072329 GB avg = 0.015 KB total ukey num = 0.069272987 Billion total entry num = 0.056080716 Billion write speed all = 0.640721040 MB/sec (with seq num) write speed all = 0.614233115 MB/sec (without seq num) 2024/11/23-13:37:30.006067 139957433374272 [compaction_job.cc:1948] [default] [JOB 26939] Generated table #261502: 930526 keys, 37705394 bytes, temperature: kWarm 2024/11/23-13:37:30.006130 139957433374272 EVENT_LOG_v1 {"time_micros": 1732340250006097, "cf_name": "default", "job": 26939, "event": "table_file_creation", "file_number": 261502, "file_size": 37705394, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3923937943, "largest_seqno": 3999056812, "table_properties": {"data_size": 31698368, "index_size": 1781888, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26054728, "raw_average_key_size": 28, "raw_value_size": 108163124, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930526, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340246, "oldest_key_time": 0, "file_creation_time": 1732340246, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261502, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:37:30.050034 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261503.sst) /dev/shm/ToplingTemp/Topling-61810-lCYLdX 2024/11/23-13:37:30.963279 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261503.sst): first pass time = 0.91's, 146.973'MB/sec 2024/11/23-13:37:32.421019 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261503.sst): old prealloc_size = 147639500, real_size = 34327200 2024/11/23-13:37:32.421598 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261503.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930243 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0323 GB avg-val = 116.28 avg-zval = 34.75 2024/11/23-13:37:32.422365 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261503.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:37:33.270194 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261503.sst): second pass time = 1.46's, 74.243'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.126'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3164.013'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 299.580'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35259.194'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 74.243'MB/sec zip pipeline throughput = 74.243'MB/sec entries = 930243 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.28 avg-zval = 34.75 usrkeys = 930243 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.28 avg-zval = 34.75 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930243 bytes = 3139584 } seq expand size = 6511701 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0323 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.5132 value = 3.3464 dict = 1.00 all = 3.5027 } Zip/UnZip{ index = 0.0951 value = 0.2988 dict = 1.00 all = 0.2855 } ---------------------------- total value len = 9.422000 GB avg = 0.165 KB (by entry num) total key len = 1.571099 GB avg = 0.028 KB total ukey len = 1.090934 GB avg = 0.016 KB total ukey num = 0.070203230 Billion total entry num = 0.057010959 Billion write speed all = 0.648902860 MB/sec (with seq num) write speed all = 0.621980826 MB/sec (without seq num) 2024/11/23-13:37:33.319652 139957433374272 [compaction_job.cc:1948] [default] [JOB 26939] Generated table #261503: 930243 keys, 38318694 bytes, temperature: kWarm 2024/11/23-13:37:33.319718 139957433374272 EVENT_LOG_v1 {"time_micros": 1732340253319684, "cf_name": "default", "job": 26939, "event": "table_file_creation", "file_number": 261503, "file_size": 38318694, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3898575090, "largest_seqno": 3999056857, "table_properties": {"data_size": 32324816, "index_size": 1769664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26046804, "raw_average_key_size": 28, "raw_value_size": 108170926, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930243, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340250, "oldest_key_time": 0, "file_creation_time": 1732340250, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261503, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:37:33.359569 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261504.sst) /dev/shm/ToplingTemp/Topling-61810-zAMHFW 2024/11/23-13:37:34.348888 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261504.sst): first pass time = 0.99's, 135.674'MB/sec 2024/11/23-13:37:35.716817 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261504.sst): old prealloc_size = 147639500, real_size = 35510832 2024/11/23-13:37:35.717345 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261504.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930508 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0335 GB avg-val = 116.24 avg-zval = 36.01 2024/11/23-13:37:35.718151 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261504.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:37:36.545523 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261504.sst): second pass time = 1.37's, 79.117'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.994'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3506.848'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.334'MB/sec write SST data time = 0.13's, 303.290'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32893.492'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.117'MB/sec zip pipeline throughput = 79.117'MB/sec entries = 930508 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.24 avg-zval = 36.01 usrkeys = 930508 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.78 avg-zval = 36.01 TagRS{ kinds = RS_Key0_TagN bytes = 145480 } TagArray{ size = 869310 bytes = 3259920 } seq expand size = 6085170 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0335 GB dict = 1.08 MB all = 0.0398 GB } UnZip/Zip{ index = 10.5132 value = 3.2280 dict = 1.00 all = 3.3747 } Zip/UnZip{ index = 0.0951 value = 0.3098 dict = 1.00 all = 0.2963 } ---------------------------- total value len = 9.536249 GB avg = 0.165 KB (by entry num) total key len = 1.597154 GB avg = 0.028 KB total ukey len = 1.109544 GB avg = 0.016 KB total ukey num = 0.071133738 Billion total entry num = 0.057941467 Billion write speed all = 0.657057655 MB/sec (with seq num) write speed all = 0.629701500 MB/sec (without seq num) 2024/11/23-13:37:36.592996 139957433374272 [compaction_job.cc:1948] [default] [JOB 26939] Generated table #261504: 930508 keys, 39771414 bytes, temperature: kWarm 2024/11/23-13:37:36.593071 139957433374272 EVENT_LOG_v1 {"time_micros": 1732340256593029, "cf_name": "default", "job": 26939, "event": "table_file_creation", "file_number": 261504, "file_size": 39771414, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3999056993, "table_properties": {"data_size": 33507872, "index_size": 1770176, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26054224, "raw_average_key_size": 28, "raw_value_size": 108163508, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930508, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340253, "oldest_key_time": 0, "file_creation_time": 1732340253, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261504, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:37:36.629706 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261505.sst) /dev/shm/ToplingTemp/Topling-61810-5lXNy0 2024/11/23-13:37:37.556424 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261505.sst): first pass time = 0.93's, 144.837'MB/sec 2024/11/23-13:37:38.866021 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261505.sst): old prealloc_size = 147639500, real_size = 32198128 2024/11/23-13:37:38.866532 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261505.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930980 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0302 GB avg-val = 116.17 avg-zval = 32.43 2024/11/23-13:37:38.867357 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261505.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:37:39.714252 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261505.sst): second pass time = 1.31's, 82.633'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 42.942'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3635.920'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.13's, 283.670'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 27086.590'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.633'MB/sec zip pipeline throughput = 82.633'MB/sec entries = 930980 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.17 avg-zval = 32.43 usrkeys = 930980 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.17 avg-zval = 32.43 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930980 bytes = 3374816 } seq expand size = 6516860 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5139 value = 3.5818 dict = 1.00 all = 3.6848 } Zip/UnZip{ index = 0.0951 value = 0.2792 dict = 1.00 all = 0.2714 } ---------------------------- total value len = 9.650916 GB avg = 0.164 KB (by entry num) total key len = 1.623221 GB avg = 0.028 KB total ukey len = 1.128164 GB avg = 0.016 KB total ukey num = 0.072064718 Billion total entry num = 0.058872447 Billion write speed all = 0.665238953 MB/sec (with seq num) write speed all = 0.637448448 MB/sec (without seq num) 2024/11/23-13:37:39.761482 139957433374272 [compaction_job.cc:1948] [default] [JOB 26939] Generated table #261505: 930980 keys, 36424652 bytes, temperature: kWarm 2024/11/23-13:37:39.761553 139957433374272 EVENT_LOG_v1 {"time_micros": 1732340259761518, "cf_name": "default", "job": 26939, "event": "table_file_creation", "file_number": 261505, "file_size": 36424652, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3509327862, "largest_seqno": 3999057160, "table_properties": {"data_size": 30194272, "index_size": 1770944, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26067440, "raw_average_key_size": 28, "raw_value_size": 108150296, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930980, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340256, "oldest_key_time": 0, "file_creation_time": 1732340256, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261505, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:37:39.798669 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261506.sst) /dev/shm/ToplingTemp/Topling-61810-nxBFo0 2024/11/23-13:37:40.824267 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261506.sst): first pass time = 1.03's, 130.872'MB/sec 2024/11/23-13:37:42.144562 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261506.sst): old prealloc_size = 147639500, real_size = 32618880 2024/11/23-13:37:42.145160 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261506.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930914 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0306 GB avg-val = 116.18 avg-zval = 32.89 2024/11/23-13:37:42.145949 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261506.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:37:42.993348 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261506.sst): second pass time = 1.32's, 81.969'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.410'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3178.860'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 290.613'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 28331.232'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.969'MB/sec zip pipeline throughput = 81.969'MB/sec entries = 930914 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.18 avg-zval = 32.89 usrkeys = 930914 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.18 avg-zval = 32.89 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930914 bytes = 3374576 } seq expand size = 6516398 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5140 value = 3.5326 dict = 1.00 all = 3.6428 } Zip/UnZip{ index = 0.0951 value = 0.2831 dict = 1.00 all = 0.2745 } ---------------------------- total value len = 9.765584 GB avg = 0.163 KB (by entry num) total key len = 1.649287 GB avg = 0.028 KB total ukey len = 1.146782 GB avg = 0.016 KB total ukey num = 0.072995632 Billion total entry num = 0.059803361 Billion write speed all = 0.673412783 MB/sec (with seq num) write speed all = 0.645188305 MB/sec (without seq num) 2024/11/23-13:37:43.042555 139957433374272 [compaction_job.cc:1948] [default] [JOB 26939] Generated table #261506: 930914 keys, 36844944 bytes, temperature: kWarm 2024/11/23-13:37:43.042627 139957433374272 EVENT_LOG_v1 {"time_micros": 1732340263042592, "cf_name": "default", "job": 26939, "event": "table_file_creation", "file_number": 261506, "file_size": 36844944, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3631595348, "largest_seqno": 3999057317, "table_properties": {"data_size": 30615152, "index_size": 1770816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26065592, "raw_average_key_size": 28, "raw_value_size": 108152172, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930914, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340259, "oldest_key_time": 0, "file_creation_time": 1732340259, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261506, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:37:43.081716 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261507.sst) /dev/shm/ToplingTemp/Topling-61810-KmFxcZ 2024/11/23-13:37:43.983995 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261507.sst): first pass time = 0.90's, 148.760'MB/sec 2024/11/23-13:37:45.331684 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261507.sst): old prealloc_size = 147639500, real_size = 34623168 2024/11/23-13:37:45.332278 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261507.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928781 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0326 GB avg-val = 116.51 avg-zval = 35.10 2024/11/23-13:37:45.333099 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261507.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:37:46.174339 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261507.sst): second pass time = 1.35's, 80.344'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.097'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3205.196'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.13's, 307.266'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33056.733'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.344'MB/sec zip pipeline throughput = 80.344'MB/sec entries = 928781 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.51 avg-zval = 35.10 usrkeys = 928781 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.51 avg-zval = 35.10 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928781 bytes = 3366848 } seq expand size = 6501467 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0326 GB dict = 1.08 MB all = 0.0388 GB } UnZip/Zip{ index = 10.3762 value = 3.3193 dict = 1.00 all = 3.4554 } Zip/UnZip{ index = 0.0964 value = 0.3013 dict = 1.00 all = 0.2894 } ---------------------------- total value len = 9.880298 GB avg = 0.163 KB (by entry num) total key len = 1.675292 GB avg = 0.028 KB total ukey len = 1.165358 GB avg = 0.016 KB total ukey num = 0.073924413 Billion total entry num = 0.060732142 Billion write speed all = 0.681586517 MB/sec (with seq num) write speed all = 0.652929074 MB/sec (without seq num) 2024/11/23-13:37:46.224037 139957433374272 [compaction_job.cc:1948] [default] [JOB 26939] Generated table #261507: 928781 keys, 38842832 bytes, temperature: kWarm 2024/11/23-13:37:46.224116 139957433374272 EVENT_LOG_v1 {"time_micros": 1732340266224069, "cf_name": "default", "job": 26939, "event": "table_file_creation", "file_number": 261507, "file_size": 38842832, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 3999057483, "table_properties": {"data_size": 32600560, "index_size": 1790208, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26005868, "raw_average_key_size": 28, "raw_value_size": 108211946, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928781, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340263, "oldest_key_time": 0, "file_creation_time": 1732340263, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261507, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:37:46.264680 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261508.sst) /dev/shm/ToplingTemp/Topling-61810-RJdBAZ 2024/11/23-13:37:47.169185 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261508.sst): first pass time = 0.90's, 148.394'MB/sec 2024/11/23-13:37:48.491260 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261508.sst): old prealloc_size = 147639500, real_size = 32164080 2024/11/23-13:37:48.491798 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261508.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931398 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0301 GB avg-val = 116.10 avg-zval = 32.37 2024/11/23-13:37:48.492645 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261508.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:37:49.356380 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261508.sst): second pass time = 1.32's, 81.845'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.067'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3450.367'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 279.771'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33379.148'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.845'MB/sec zip pipeline throughput = 81.845'MB/sec entries = 931398 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.10 avg-zval = 32.37 usrkeys = 931398 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.10 avg-zval = 32.37 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931398 bytes = 3376336 } seq expand size = 6519786 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.4454 value = 3.5870 dict = 1.00 all = 3.6882 } Zip/UnZip{ index = 0.0957 value = 0.2788 dict = 1.00 all = 0.2711 } ---------------------------- total value len = 9.994956 GB avg = 0.162 KB (by entry num) total key len = 1.701372 GB avg = 0.028 KB total ukey len = 1.183986 GB avg = 0.016 KB total ukey num = 0.074855811 Billion total entry num = 0.061663540 Billion write speed all = 0.689758218 MB/sec (with seq num) write speed all = 0.660666740 MB/sec (without seq num) 2024/11/23-13:37:49.402066 139957433374272 [compaction_job.cc:1948] [default] [JOB 26939] Generated table #261508: 931398 keys, 36391576 bytes, temperature: kWarm 2024/11/23-13:37:49.402134 139957433374272 EVENT_LOG_v1 {"time_micros": 1732340269402100, "cf_name": "default", "job": 26939, "event": "table_file_creation", "file_number": 261508, "file_size": 36391576, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3621075956, "largest_seqno": 3999057363, "table_properties": {"data_size": 30147680, "index_size": 1783360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26079144, "raw_average_key_size": 28, "raw_value_size": 108138596, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931398, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340266, "oldest_key_time": 0, "file_creation_time": 1732340266, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261508, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:37:49.439095 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261509.sst) /dev/shm/ToplingTemp/Topling-61810-X7NSv0 2024/11/23-13:37:50.283769 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261509.sst): first pass time = 0.84's, 158.906'MB/sec 2024/11/23-13:37:51.607649 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261509.sst): old prealloc_size = 147639500, real_size = 32436720 2024/11/23-13:37:51.608230 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261509.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932719 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0304 GB avg-val = 115.90 avg-zval = 32.61 2024/11/23-13:37:51.609741 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261509.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:37:52.461678 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261509.sst): second pass time = 1.32's, 81.704'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.074'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3186.797'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.321'MB/sec write SST data time = 0.13's, 288.346'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33042.447'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.704'MB/sec zip pipeline throughput = 81.704'MB/sec entries = 932719 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.90 avg-zval = 32.61 usrkeys = 932719 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.90 avg-zval = 32.61 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932719 bytes = 3497712 } seq expand size = 6529033 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.4464 value = 3.5539 dict = 1.00 all = 3.6487 } Zip/UnZip{ index = 0.0957 value = 0.2814 dict = 1.00 all = 0.2741 } ---------------------------- total value len = 10.109587 GB avg = 0.162 KB (by entry num) total key len = 1.727488 GB avg = 0.028 KB total ukey len = 1.202640 GB avg = 0.016 KB total ukey num = 0.075788530 Billion total entry num = 0.062596259 Billion write speed all = 0.697930563 MB/sec (with seq num) write speed all = 0.668404456 MB/sec (without seq num) 2024/11/23-13:37:52.507394 139957433374272 [compaction_job.cc:1948] [default] [JOB 26939] Generated table #261509: 932719 keys, 36785166 bytes, temperature: kWarm 2024/11/23-13:37:52.507461 139957433374272 EVENT_LOG_v1 {"time_micros": 1732340272507426, "cf_name": "default", "job": 26939, "event": "table_file_creation", "file_number": 261509, "file_size": 36785166, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3999057421, "table_properties": {"data_size": 30417632, "index_size": 1785728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26116132, "raw_average_key_size": 28, "raw_value_size": 108101722, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932719, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340269, "oldest_key_time": 0, "file_creation_time": 1732340269, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261509, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:37:52.543560 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261510.sst) /dev/shm/ToplingTemp/Topling-61810-KxTywY 2024/11/23-13:37:52.656941 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261510.sst): first pass time = 0.11's, 238.097'MB/sec 2024/11/23-13:37:52.920510 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261510.sst): old prealloc_size = 147639500, real_size = 7193120 2024/11/23-13:37:52.920688 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261510.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 189837 prefix = 4 raw-key = 0.0030 GB zip-key = 0.0004 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0217 GB zip-val = 0.0068 GB avg-val = 114.14 avg-zval = 35.73 2024/11/23-13:37:52.920902 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261510.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0004 GB, waited 0.000 sec, Key+Value = 0.027 GB 2024/11/23-13:37:53.088456 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00/261510.sst): second pass time = 0.26's, 82.441'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 48.017'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2342.296'MB/sec (index lex order gen) rebuild zvType time = 0.14's, 0.336'MB/sec write SST data time = 0.03's, 305.410'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.227'MB, throughput = 8544.075'MB/sec zip my value time = 0.26's, unzip length = 0.022'GB zip my value throughput = 82.441'MB/sec zip pipeline throughput = 82.441'MB/sec entries = 189837 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.14 avg-zval = 35.73 usrkeys = 189837 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.14 avg-zval = 35.73 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 189837 bytes = 688176 } seq expand size = 1328859 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0038 GB value = 0.0217 GB dict = 0.23 MB all = 0.0270 GB } __ZipSize{ index = 0.0004 GB value = 0.0068 GB dict = 0.23 MB all = 0.0081 GB } UnZip/Zip{ index = 10.4739 value = 3.1946 dict = 1.00 all = 3.3465 } Zip/UnZip{ index = 0.0955 value = 0.3130 dict = 1.00 all = 0.2988 } ---------------------------- total value len = 10.132585 GB avg = 0.161 KB (by entry num) total key len = 1.732803 GB avg = 0.028 KB total ukey len = 1.206437 GB avg = 0.016 KB total ukey num = 0.075978367 Billion total entry num = 0.062786096 Billion write speed all = 0.699574083 MB/sec (with seq num) write speed all = 0.669959526 MB/sec (without seq num) 2024/11/23-13:37:53.104566 139957433374272 [compaction_job.cc:1948] [default] [JOB 26939] Generated table #261510: 189837 keys, 8063311 bytes, temperature: kWarm 2024/11/23-13:37:53.104640 139957433374272 EVENT_LOG_v1 {"time_micros": 1732340273104607, "cf_name": "default", "job": 26939, "event": "table_file_creation", "file_number": 261510, "file_size": 8063311, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3563652023, "largest_seqno": 3999044275, "table_properties": {"data_size": 6782992, "index_size": 362496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 5315436, "raw_average_key_size": 28, "raw_value_size": 21668694, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 189837, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340272, "oldest_key_time": 0, "file_creation_time": 1732340272, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261510, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:37:53.120876 139957433374272 [compaction_job.cc:703] job-26939: subcompact[0], size: 0.418856 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26939/att-00 [261499,261500,261501,261502,261503,261504,261505,261506,261507,261508,261509,261510] 2024/11/23-13:37:53.195235 139957433374272 (Original Log Time 2024/11/23-13:37:15.394479) EVENT_LOG_v1 {"time_micros": 1732340235394454, "job": 26939, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [261426], "files_L3": [261376, 261377, 261378, 261379, 261380, 261381, 261382, 261383, 261384, 261385, 261482], "score": 0.988809, "input_data_size": 440704882, "oldest_snapshot_seqno": 3999400600} 2024/11/23-13:37:53.195238 139957433374272 (Original Log Time 2024/11/23-13:37:53.194218) [compaction_job.cc:2022] [default] [JOB 26939] Compacted 1@2 + 11@3 files to L3 => 418856325 bytes 2024/11/23-13:37:53.195240 139957433374272 (Original Log Time 2024/11/23-13:37:53.195168) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 12 0 0 0] max score 0.02, MB/sec: 11.7 rd, 11.1 wr, level 3, files in(1, 11) out(12 +0 blob) MB in(27.1, 393.2 +0.0 blob) out(399.5 +0.0 blob), read-write-amplify(30.2) write-amplify(14.7) OK, records in: 10473961, records dropped: 46029 output_compression: Snappy 2024/11/23-13:37:53.195246 139957433374272 (Original Log Time 2024/11/23-13:37:53.195214) EVENT_LOG_v1 {"time_micros": 1732340273195193, "cf": "default", "job": 26939, "event": "compaction_finished", "compaction_time_micros": 37726405, "compaction_time_cpu_micros": 16170371, "output_level": 3, "num_output_files": 12, "total_output_size": 418856325, "num_input_records": 10473961, "num_output_records": 10427932, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 311276385, "file_range_sync_nanos": 0, "file_fsync_nanos": 8842, "file_prepare_write_nanos": 31368, "lsm_state": [0, 0, 0, 12, 0, 0, 0]} 2024/11/23-13:37:53.327301 139957433374272 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732340235,"executesMs":37952,"compactionJobId":26939,"attempt":0,"compactionInputRawBytes":1509915850,"compactionInputZipBytes":440704882,"compactionOutputRawBytes":0,"compactionOutputZipBytes":418856325,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-3250ceb795c46be6108327d6abd82054-b975a820ceb09631-00"}}, response = 2024/11/23-13:37:53.327398 139957433374272 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-26939/att-00: olev 3, work 37.953 s, result 7.761 ms, install 1.044 ms, input{raw 1.406 GiB zip 420.289 MiB}