2024/11/23-00:17:26.212582 140024886109760 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00: fork to child time = 0.009409 sec 2024/11/23-00:17:26.213403 140024886109760 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00 2024/11/23-00:17:26.220841 140024886109760 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00 2024/11/23-00:17:26.222013 140024886109760 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-00:17:26.222298 140024886109760 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-00:17:26.278568 140024886109760 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-15004/att-00/MANIFEST-206347 2024/11/23-00:17:26.292749 140024886109760 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-15004/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 235892, last_sequence is 3669509140, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 235744 2024/11/23-00:17:26.292755 140024886109760 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-00:17:26.292844 140024886109760 [version_set.cc:5721] Creating manifest 235892 2024/11/23-00:17:26.332016 140024886109760 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-00:17:26.352692 140024886109760 [compaction_job.cc:2437] [default] [JOB 15004] Compacting 1@2 + 13@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-00:17:26.352710 140024886109760 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 11243 Base level 2, inputs: [235826(27MB)], [235878(6456KB) 235794(36MB) 235795(10MB) 235771(35MB) 235772(36MB) 235773(35MB) 235774(36MB) 235775(35MB) 235776(34MB) 235777(35MB) 235778(36MB) 235780(34MB) 235844(35MB)] 2024/11/23-00:17:26.358708 140024886109760 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-00:17:26.358741 140024886109760 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-00:17:26.358758 140024886109760 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-00:17:26.358763 140024886109760 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-00:17:26.358774 140024886109760 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-00:17:26.614732 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235893.sst) /dev/shm/ToplingTemp/Topling-347295-q5j9YZ 2024/11/23-00:17:27.544202 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235893.sst): first pass time = 0.93's, 144.409'MB/sec 2024/11/23-00:17:29.001054 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235893.sst): old prealloc_size = 147639500, real_size = 33437440 2024/11/23-00:17:29.001642 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235893.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929716 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0314 GB avg-val = 116.36 avg-zval = 33.81 2024/11/23-00:17:29.002478 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235893.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:17:29.939544 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235893.sst): second pass time = 1.46's, 74.348'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 40.129'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3111.929'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.288'MB/sec write SST data time = 0.13's, 286.036'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32900.037'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 74.348'MB/sec zip pipeline throughput = 74.348'MB/sec entries = 929716 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.36 avg-zval = 33.81 usrkeys = 929716 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.36 avg-zval = 33.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929716 bytes = 3370240 } seq expand size = 6508012 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5133 value = 3.4414 dict = 1.00 all = 3.5639 } Zip/UnZip{ index = 0.0951 value = 0.2906 dict = 1.00 all = 0.2806 } ---------------------------- total value len = 0.114694 GB avg = 0.123 KB (by entry num) total key len = 0.026032 GB avg = 0.028 KB total ukey len = 0.018594 GB avg = 0.020 KB total ukey num = 0.000929716 Billion total entry num = 0.000929716 Billion write speed all = 0.002535287 MB/sec (with seq num) write speed all = 0.002401290 MB/sec (without seq num) 2024/11/23-00:17:29.972587 140024886109760 [compaction_job.cc:1948] [default] [JOB 15004] Generated table #235893: 929716 keys, 37660652 bytes, temperature: kWarm 2024/11/23-00:17:29.972681 140024886109760 EVENT_LOG_v1 {"time_micros": 1732292249972626, "cf_name": "default", "job": 15004, "event": "table_file_creation", "file_number": 235893, "file_size": 37660652, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3668864346, "table_properties": {"data_size": 31436208, "index_size": 1768640, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26032048, "raw_average_key_size": 28, "raw_value_size": 108185708, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929716, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732292246, "oldest_key_time": 0, "file_creation_time": 1732292246, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 235893, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:17:30.008357 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235894.sst) /dev/shm/ToplingTemp/Topling-347295-ItGLaZ 2024/11/23-00:17:30.876790 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235894.sst): first pass time = 0.87's, 154.560'MB/sec 2024/11/23-00:17:32.385377 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235894.sst): old prealloc_size = 147639500, real_size = 33658352 2024/11/23-00:17:32.385941 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235894.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927438 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0317 GB avg-val = 116.72 avg-zval = 34.13 2024/11/23-00:17:32.386724 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235894.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:17:33.334397 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235894.sst): second pass time = 1.51's, 71.793'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.910'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3298.748'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.284'MB/sec write SST data time = 0.13's, 287.560'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30587.096'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.793'MB/sec zip pipeline throughput = 92.554'MB/sec entries = 927438 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.72 avg-zval = 34.13 usrkeys = 927438 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.72 avg-zval = 34.13 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927438 bytes = 3361984 } seq expand size = 6492066 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.4441 value = 3.4202 dict = 1.00 all = 3.5437 } Zip/UnZip{ index = 0.0957 value = 0.2924 dict = 1.00 all = 0.2822 } ---------------------------- total value len = 0.229435 GB avg = 0.124 KB (by entry num) total key len = 0.052000 GB avg = 0.028 KB total ukey len = 0.037143 GB avg = 0.020 KB total ukey num = 0.001857154 Billion total entry num = 0.001857154 Billion write speed all = 0.005069977 MB/sec (with seq num) write speed all = 0.004802329 MB/sec (without seq num) 2024/11/23-00:17:33.368170 140024886109760 [compaction_job.cc:1948] [default] [JOB 15004] Generated table #235894: 927438 keys, 37874748 bytes, temperature: kWarm 2024/11/23-00:17:33.368237 140024886109760 EVENT_LOG_v1 {"time_micros": 1732292253368203, "cf_name": "default", "job": 15004, "event": "table_file_creation", "file_number": 235894, "file_size": 37874748, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3326330223, "largest_seqno": 3668864680, "table_properties": {"data_size": 31650336, "index_size": 1776000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25968264, "raw_average_key_size": 28, "raw_value_size": 108249544, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927438, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732292249, "oldest_key_time": 0, "file_creation_time": 1732292249, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 235894, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:17:33.405117 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235895.sst) /dev/shm/ToplingTemp/Topling-347295-i7wqyZ 2024/11/23-00:17:34.235731 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235895.sst): first pass time = 0.83's, 161.596'MB/sec 2024/11/23-00:17:35.797482 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235895.sst): old prealloc_size = 147639500, real_size = 33000656 2024/11/23-00:17:35.798033 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235895.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925968 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 116.95 avg-zval = 33.49 2024/11/23-00:17:35.798804 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235895.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:17:36.718250 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235895.sst): second pass time = 1.56's, 69.400'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.164'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3346.930'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.293'MB/sec write SST data time = 0.13's, 284.577'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 29789.796'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.400'MB/sec zip pipeline throughput = 115.247'MB/sec entries = 925968 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.95 avg-zval = 33.49 usrkeys = 925968 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.95 avg-zval = 33.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925968 bytes = 3125152 } seq expand size = 6481776 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5132 value = 3.4924 dict = 1.00 all = 3.6294 } Zip/UnZip{ index = 0.0951 value = 0.2863 dict = 1.00 all = 0.2755 } ---------------------------- total value len = 0.344208 GB avg = 0.124 KB (by entry num) total key len = 0.077927 GB avg = 0.028 KB total ukey len = 0.055662 GB avg = 0.020 KB total ukey num = 0.002783122 Billion total entry num = 0.002783122 Billion write speed all = 0.007604173 MB/sec (with seq num) write speed all = 0.007203101 MB/sec (without seq num) 2024/11/23-00:17:36.756719 140024886109760 [compaction_job.cc:1948] [default] [JOB 15004] Generated table #235895: 925968 keys, 36980856 bytes, temperature: kWarm 2024/11/23-00:17:36.756792 140024886109760 EVENT_LOG_v1 {"time_micros": 1732292256756751, "cf_name": "default", "job": 15004, "event": "table_file_creation", "file_number": 235895, "file_size": 36980856, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3554731625, "largest_seqno": 3668864277, "table_properties": {"data_size": 31007424, "index_size": 1761536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25927104, "raw_average_key_size": 28, "raw_value_size": 108290660, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925968, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732292253, "oldest_key_time": 0, "file_creation_time": 1732292253, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 235895, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:17:36.798126 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235896.sst) /dev/shm/ToplingTemp/Topling-347295-O8movZ 2024/11/23-00:17:37.669855 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235896.sst): first pass time = 0.87's, 153.975'MB/sec 2024/11/23-00:17:39.008612 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235896.sst): old prealloc_size = 147639500, real_size = 32546288 2024/11/23-00:17:39.009214 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235896.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930719 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0305 GB avg-val = 116.21 avg-zval = 32.82 2024/11/23-00:17:39.009997 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235896.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:17:39.952917 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235896.sst): second pass time = 1.34's, 80.838'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 38.818'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3047.089'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.286'MB/sec write SST data time = 0.13's, 281.664'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 29684.019'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.838'MB/sec zip pipeline throughput = 80.838'MB/sec entries = 930719 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.21 avg-zval = 32.82 usrkeys = 930719 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.21 avg-zval = 32.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930719 bytes = 3257536 } seq expand size = 6515033 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5144 value = 3.5412 dict = 1.00 all = 3.6616 } Zip/UnZip{ index = 0.0951 value = 0.2824 dict = 1.00 all = 0.2731 } ---------------------------- total value len = 0.458880 GB avg = 0.124 KB (by entry num) total key len = 0.103988 GB avg = 0.028 KB total ukey len = 0.074277 GB avg = 0.020 KB total ukey num = 0.003713841 Billion total entry num = 0.003713841 Billion write speed all = 0.010138686 MB/sec (with seq num) write speed all = 0.009603521 MB/sec (without seq num) 2024/11/23-00:17:39.985385 140024886109760 [compaction_job.cc:1948] [default] [JOB 15004] Generated table #235896: 930719 keys, 36655312 bytes, temperature: kWarm 2024/11/23-00:17:39.985449 140024886109760 EVENT_LOG_v1 {"time_micros": 1732292259985417, "cf_name": "default", "job": 15004, "event": "table_file_creation", "file_number": 235896, "file_size": 36655312, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 3668862929, "table_properties": {"data_size": 30543072, "index_size": 1770368, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26060132, "raw_average_key_size": 28, "raw_value_size": 108157654, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930719, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732292256, "oldest_key_time": 0, "file_creation_time": 1732292256, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 235896, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:17:40.023058 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235897.sst) /dev/shm/ToplingTemp/Topling-347295-fUidi1 2024/11/23-00:17:40.942045 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235897.sst): first pass time = 0.92's, 146.055'MB/sec 2024/11/23-00:17:42.271956 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235897.sst): old prealloc_size = 147639500, real_size = 31811584 2024/11/23-00:17:42.272536 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235897.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925638 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0298 GB avg-val = 117.00 avg-zval = 32.21 2024/11/23-00:17:42.273345 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235897.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:17:43.165398 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235897.sst): second pass time = 1.33's, 81.479'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 40.319'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3233.399'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.304'MB/sec write SST data time = 0.13's, 273.301'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32329.793'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.479'MB/sec zip pipeline throughput = 81.479'MB/sec entries = 925638 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.00 avg-zval = 32.21 usrkeys = 925638 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.00 avg-zval = 32.21 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925638 bytes = 3355456 } seq expand size = 6479466 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0298 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.5133 value = 3.6319 dict = 1.00 all = 3.7260 } Zip/UnZip{ index = 0.0951 value = 0.2753 dict = 1.00 all = 0.2684 } ---------------------------- total value len = 0.573660 GB avg = 0.124 KB (by entry num) total key len = 0.129905 GB avg = 0.028 KB total ukey len = 0.092790 GB avg = 0.020 KB total ukey num = 0.004639479 Billion total entry num = 0.004639479 Billion write speed all = 0.012672268 MB/sec (with seq num) write speed all = 0.012003756 MB/sec (without seq num) 2024/11/23-00:17:43.198369 140024886109760 [compaction_job.cc:1948] [default] [JOB 15004] Generated table #235897: 925638 keys, 36022014 bytes, temperature: kWarm 2024/11/23-00:17:43.198435 140024886109760 EVENT_LOG_v1 {"time_micros": 1732292263198402, "cf_name": "default", "job": 15004, "event": "table_file_creation", "file_number": 235897, "file_size": 36022014, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 3668863614, "table_properties": {"data_size": 29819120, "index_size": 1760896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25917864, "raw_average_key_size": 28, "raw_value_size": 108299880, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925638, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732292260, "oldest_key_time": 0, "file_creation_time": 1732292260, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 235897, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:17:43.236248 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235898.sst) /dev/shm/ToplingTemp/Topling-347295-0O4XLZ 2024/11/23-00:17:44.037414 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235898.sst): first pass time = 0.80's, 167.537'MB/sec 2024/11/23-00:17:45.356573 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235898.sst): old prealloc_size = 147639500, real_size = 34023568 2024/11/23-00:17:45.357141 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235898.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927271 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0320 GB avg-val = 116.74 avg-zval = 34.52 2024/11/23-00:17:45.357820 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235898.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:17:46.240869 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235898.sst): second pass time = 1.32's, 82.105'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 42.661'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3270.846'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.308'MB/sec write SST data time = 0.13's, 287.110'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34881.813'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.105'MB/sec zip pipeline throughput = 82.105'MB/sec entries = 927271 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.74 avg-zval = 34.52 usrkeys = 927271 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.36 avg-zval = 34.52 TagRS{ kinds = RS_Key0_TagN bytes = 145000 } TagArray{ size = 744274 bytes = 2698016 } seq expand size = 5209918 multi value expand size = 0 cnt WriteAppend = 19 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.3794 value = 3.3824 dict = 1.00 all = 3.5578 } Zip/UnZip{ index = 0.0963 value = 0.2956 dict = 1.00 all = 0.2811 } ---------------------------- total value len = 0.687124 GB avg = 0.123 KB (by entry num) total key len = 0.155869 GB avg = 0.028 KB total ukey len = 0.111335 GB avg = 0.020 KB total ukey num = 0.005566750 Billion total entry num = 0.005566750 Billion write speed all = 0.015182729 MB/sec (with seq num) write speed all = 0.014380649 MB/sec (without seq num) 2024/11/23-00:17:46.274446 140024886109760 [compaction_job.cc:1948] [default] [JOB 15004] Generated table #235898: 927271 keys, 37724524 bytes, temperature: kWarm 2024/11/23-00:17:46.274506 140024886109760 EVENT_LOG_v1 {"time_micros": 1732292266274474, "cf_name": "default", "job": 15004, "event": "table_file_creation", "file_number": 235898, "file_size": 37724524, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3668864411, "table_properties": {"data_size": 32004800, "index_size": 1786752, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25963588, "raw_average_key_size": 28, "raw_value_size": 108254146, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927271, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732292263, "oldest_key_time": 0, "file_creation_time": 1732292263, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 235898, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:17:46.310846 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235899.sst) /dev/shm/ToplingTemp/Topling-347295-LdOiQ0 2024/11/23-00:17:47.161557 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235899.sst): first pass time = 0.85's, 157.778'MB/sec 2024/11/23-00:17:48.510496 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235899.sst): old prealloc_size = 147639500, real_size = 32463312 2024/11/23-00:17:48.511063 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235899.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922492 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0305 GB avg-val = 117.49 avg-zval = 33.04 2024/11/23-00:17:48.511843 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235899.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:17:49.410602 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235899.sst): second pass time = 1.35's, 80.395'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 40.607'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3227.584'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.299'MB/sec write SST data time = 0.13's, 284.460'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32749.395'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.395'MB/sec zip pipeline throughput = 80.395'MB/sec entries = 922492 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.49 avg-zval = 33.04 usrkeys = 922492 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.49 avg-zval = 33.04 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922492 bytes = 3228736 } seq expand size = 6457444 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5131 value = 3.5563 dict = 1.00 all = 3.6721 } Zip/UnZip{ index = 0.0951 value = 0.2812 dict = 1.00 all = 0.2723 } ---------------------------- total value len = 0.801969 GB avg = 0.124 KB (by entry num) total key len = 0.181699 GB avg = 0.028 KB total ukey len = 0.129785 GB avg = 0.020 KB total ukey num = 0.006489242 Billion total entry num = 0.006489242 Billion write speed all = 0.017715350 MB/sec (with seq num) write speed all = 0.016780407 MB/sec (without seq num) 2024/11/23-00:17:49.449220 140024886109760 [compaction_job.cc:1948] [default] [JOB 15004] Generated table #235899: 922492 keys, 36550368 bytes, temperature: kWarm 2024/11/23-00:17:49.449283 140024886109760 EVENT_LOG_v1 {"time_micros": 1732292269449251, "cf_name": "default", "job": 15004, "event": "table_file_creation", "file_number": 235899, "file_size": 36550368, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 3668862154, "table_properties": {"data_size": 30477568, "index_size": 1754944, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25829776, "raw_average_key_size": 28, "raw_value_size": 108388012, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922492, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732292266, "oldest_key_time": 0, "file_creation_time": 1732292266, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 235899, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:17:49.485587 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235900.sst) /dev/shm/ToplingTemp/Topling-347295-voSTJ0 2024/11/23-00:17:50.397434 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235900.sst): first pass time = 0.91's, 147.199'MB/sec 2024/11/23-00:17:51.749512 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235900.sst): old prealloc_size = 147639500, real_size = 33327008 2024/11/23-00:17:51.750055 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235900.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925953 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.95 avg-zval = 33.84 2024/11/23-00:17:51.750806 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235900.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:17:52.697273 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235900.sst): second pass time = 1.35's, 80.131'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.486'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3403.353'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.282'MB/sec write SST data time = 0.13's, 294.241'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30959.995'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.131'MB/sec zip pipeline throughput = 80.131'MB/sec entries = 925953 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.95 avg-zval = 33.84 usrkeys = 925953 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.95 avg-zval = 33.84 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925953 bytes = 3240848 } seq expand size = 6481671 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5142 value = 3.4560 dict = 1.00 all = 3.5865 } Zip/UnZip{ index = 0.0951 value = 0.2893 dict = 1.00 all = 0.2788 } ---------------------------- total value len = 0.916742 GB avg = 0.124 KB (by entry num) total key len = 0.207625 GB avg = 0.028 KB total ukey len = 0.148304 GB avg = 0.020 KB total ukey num = 0.007415195 Billion total entry num = 0.007415195 Billion write speed all = 0.020248075 MB/sec (with seq num) write speed all = 0.019179787 MB/sec (without seq num) 2024/11/23-00:17:52.729045 140024886109760 [compaction_job.cc:1948] [default] [JOB 15004] Generated table #235900: 925953 keys, 37422540 bytes, temperature: kWarm 2024/11/23-00:17:52.729109 140024886109760 EVENT_LOG_v1 {"time_micros": 1732292272729077, "cf_name": "default", "job": 15004, "event": "table_file_creation", "file_number": 235900, "file_size": 37422540, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 3668864422, "table_properties": {"data_size": 31333968, "index_size": 1761344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25926684, "raw_average_key_size": 28, "raw_value_size": 108291054, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925953, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732292269, "oldest_key_time": 0, "file_creation_time": 1732292269, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 235900, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:17:52.765473 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235901.sst) /dev/shm/ToplingTemp/Topling-347295-UHNp80 2024/11/23-00:17:53.594936 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235901.sst): first pass time = 0.83's, 161.821'MB/sec 2024/11/23-00:17:54.939543 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235901.sst): old prealloc_size = 147639500, real_size = 33196240 2024/11/23-00:17:54.940113 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235901.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932343 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0312 GB avg-val = 115.96 avg-zval = 33.44 2024/11/23-00:17:54.940892 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235901.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:17:55.867349 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235901.sst): second pass time = 1.34's, 80.444'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.214'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3272.285'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.292'MB/sec write SST data time = 0.13's, 291.423'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33410.218'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.444'MB/sec zip pipeline throughput = 80.444'MB/sec entries = 932343 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.96 avg-zval = 33.44 usrkeys = 932343 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.96 avg-zval = 33.44 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932343 bytes = 3263216 } seq expand size = 6526401 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4440 value = 3.4676 dict = 1.00 all = 3.5974 } Zip/UnZip{ index = 0.0957 value = 0.2884 dict = 1.00 all = 0.2780 } ---------------------------- total value len = 1.031381 GB avg = 0.124 KB (by entry num) total key len = 0.233731 GB avg = 0.028 KB total ukey len = 0.166951 GB avg = 0.020 KB total ukey num = 0.008347538 Billion total entry num = 0.008347538 Billion write speed all = 0.022781355 MB/sec (with seq num) write speed all = 0.021578816 MB/sec (without seq num) 2024/11/23-00:17:55.898629 140024886109760 [compaction_job.cc:1948] [default] [JOB 15004] Generated table #235901: 932343 keys, 37309888 bytes, temperature: kWarm 2024/11/23-00:17:55.898693 140024886109760 EVENT_LOG_v1 {"time_micros": 1732292275898662, "cf_name": "default", "job": 15004, "event": "table_file_creation", "file_number": 235901, "file_size": 37309888, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3485447251, "largest_seqno": 3668864458, "table_properties": {"data_size": 31177536, "index_size": 1785408, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26105604, "raw_average_key_size": 28, "raw_value_size": 108112266, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932343, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732292272, "oldest_key_time": 0, "file_creation_time": 1732292272, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 235901, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:17:55.936226 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235902.sst) /dev/shm/ToplingTemp/Topling-347295-sOJ16Z 2024/11/23-00:17:56.795886 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235902.sst): first pass time = 0.86's, 156.136'MB/sec 2024/11/23-00:17:58.139020 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235902.sst): old prealloc_size = 147639500, real_size = 34385712 2024/11/23-00:17:58.139577 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235902.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929894 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0324 GB avg-val = 116.34 avg-zval = 34.83 2024/11/23-00:17:58.140334 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235902.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:17:59.054100 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235902.sst): second pass time = 1.34's, 80.617'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 42.065'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3290.595'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.296'MB/sec write SST data time = 0.13's, 297.372'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 27968.424'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.617'MB/sec zip pipeline throughput = 80.617'MB/sec entries = 929894 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.34 avg-zval = 34.83 usrkeys = 929894 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.34 avg-zval = 34.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929894 bytes = 3254640 } seq expand size = 6509258 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0324 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.5123 value = 3.3406 dict = 1.00 all = 3.4869 } Zip/UnZip{ index = 0.0951 value = 0.2993 dict = 1.00 all = 0.2868 } ---------------------------- total value len = 1.146071 GB avg = 0.124 KB (by entry num) total key len = 0.259768 GB avg = 0.028 KB total ukey len = 0.185549 GB avg = 0.020 KB total ukey num = 0.009277432 Billion total entry num = 0.009277432 Billion write speed all = 0.025314030 MB/sec (with seq num) write speed all = 0.023977608 MB/sec (without seq num) 2024/11/23-00:17:59.087379 140024886109760 [compaction_job.cc:1948] [default] [JOB 15004] Generated table #235902: 929894 keys, 38492566 bytes, temperature: kWarm 2024/11/23-00:17:59.087442 140024886109760 EVENT_LOG_v1 {"time_micros": 1732292279087411, "cf_name": "default", "job": 15004, "event": "table_file_creation", "file_number": 235902, "file_size": 38492566, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268533, "largest_seqno": 3668864489, "table_properties": {"data_size": 32383904, "index_size": 1769152, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26037032, "raw_average_key_size": 28, "raw_value_size": 108180832, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929894, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732292275, "oldest_key_time": 0, "file_creation_time": 1732292275, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 235902, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:17:59.129253 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235903.sst) /dev/shm/ToplingTemp/Topling-347295-P11EWZ 2024/11/23-00:18:00.052043 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235903.sst): first pass time = 0.92's, 145.454'MB/sec 2024/11/23-00:18:01.365951 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235903.sst): old prealloc_size = 147639500, real_size = 33283136 2024/11/23-00:18:01.366500 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235903.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926215 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.91 avg-zval = 33.78 2024/11/23-00:18:01.367240 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235903.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:18:02.304552 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235903.sst): second pass time = 1.31's, 82.458'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.703'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3341.776'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.286'MB/sec write SST data time = 0.13's, 289.280'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33633.528'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.458'MB/sec zip pipeline throughput = 82.458'MB/sec entries = 926215 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.91 avg-zval = 33.78 usrkeys = 926215 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.91 avg-zval = 33.78 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926215 bytes = 3125984 } seq expand size = 6483505 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5103 value = 3.4608 dict = 1.00 all = 3.6018 } Zip/UnZip{ index = 0.0951 value = 0.2890 dict = 1.00 all = 0.2776 } ---------------------------- total value len = 1.260838 GB avg = 0.124 KB (by entry num) total key len = 0.285702 GB avg = 0.028 KB total ukey len = 0.204073 GB avg = 0.020 KB total ukey num = 0.010203647 Billion total entry num = 0.010203647 Billion write speed all = 0.027845918 MB/sec (with seq num) write speed all = 0.026376161 MB/sec (without seq num) 2024/11/23-00:18:02.336663 140024886109760 [compaction_job.cc:1948] [default] [JOB 15004] Generated table #235903: 926215 keys, 37263954 bytes, temperature: kWarm 2024/11/23-00:18:02.336744 140024886109760 EVENT_LOG_v1 {"time_micros": 1732292282336702, "cf_name": "default", "job": 15004, "event": "table_file_creation", "file_number": 235903, "file_size": 37263954, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3596991077, "largest_seqno": 3668864519, "table_properties": {"data_size": 31288880, "index_size": 1762496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25934020, "raw_average_key_size": 28, "raw_value_size": 108283834, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926215, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732292279, "oldest_key_time": 0, "file_creation_time": 1732292279, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 235903, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:18:02.390825 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235904.sst) /dev/shm/ToplingTemp/Topling-347295-YZk8d1 2024/11/23-00:18:02.713558 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235904.sst): first pass time = 0.32's, 243.668'MB/sec 2024/11/23-00:18:03.499712 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235904.sst): old prealloc_size = 147639500, real_size = 20665280 2024/11/23-00:18:03.500084 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235904.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 544830 prefix = 4 raw-key = 0.0087 GB zip-key = 0.0010 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0634 GB zip-val = 0.0195 GB avg-val = 116.32 avg-zval = 35.78 2024/11/23-00:18:03.500568 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235904.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0014 GB, waited 0.000 sec, Key+Value = 0.079 GB 2024/11/23-00:18:04.042576 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00/235904.sst): second pass time = 0.79's, 80.682'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.54's, 42.510'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2989.160'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.292'MB/sec write SST data time = 0.08's, 302.484'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.637'MB, throughput = 19295.904'MB/sec zip my value time = 0.79's, unzip length = 0.063'GB zip my value throughput = 80.682'MB/sec zip pipeline throughput = 80.682'MB/sec entries = 544830 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.32 avg-zval = 35.78 usrkeys = 544830 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.32 avg-zval = 35.78 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 544830 bytes = 1906912 } seq expand size = 3813810 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0109 GB value = 0.0634 GB dict = 0.64 MB all = 0.0786 GB } __ZipSize{ index = 0.0010 GB value = 0.0195 GB dict = 0.64 MB all = 0.0231 GB } UnZip/Zip{ index = 10.5040 value = 3.2513 dict = 1.00 all = 3.4072 } Zip/UnZip{ index = 0.0952 value = 0.3076 dict = 1.00 all = 0.2935 } ---------------------------- total value len = 1.328025 GB avg = 0.124 KB (by entry num) total key len = 0.300957 GB avg = 0.028 KB total ukey len = 0.214970 GB avg = 0.020 KB total ukey num = 0.010748477 Billion total entry num = 0.010748477 Billion write speed all = 0.029329387 MB/sec (with seq num) write speed all = 0.027781199 MB/sec (without seq num) 2024/11/23-00:18:04.068517 140024886109760 [compaction_job.cc:1948] [default] [JOB 15004] Generated table #235904: 544830 keys, 23076729 bytes, temperature: kWarm 2024/11/23-00:18:04.068578 140024886109760 EVENT_LOG_v1 {"time_micros": 1732292284068546, "cf_name": "default", "job": 15004, "event": "table_file_creation", "file_number": 235904, "file_size": 23076729, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 3668863580, "table_properties": {"data_size": 19491504, "index_size": 1037376, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 15255240, "raw_average_key_size": 28, "raw_value_size": 63372624, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 544830, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732292282, "oldest_key_time": 0, "file_creation_time": 1732292282, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 235904, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:18:04.093167 140024886109760 [compaction_job.cc:703] job-15004: subcompact[0], size: 0.433034 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15004/att-00 [235893,235894,235895,235896,235897,235898,235899,235900,235901,235902,235903,235904] 2024/11/23-00:18:04.168014 140024886109760 (Original Log Time 2024/11/23-00:17:26.352750) EVENT_LOG_v1 {"time_micros": 1732292246352720, "job": 15004, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [235826], "files_L3": [235878, 235794, 235795, 235771, 235772, 235773, 235774, 235775, 235776, 235777, 235778, 235780, 235844], "score": 0.999776, "input_data_size": 457512940, "oldest_snapshot_seqno": 3669509135} 2024/11/23-00:18:04.168017 140024886109760 (Original Log Time 2024/11/23-00:18:04.167335) [compaction_job.cc:2022] [default] [JOB 15004] Compacted 1@2 + 13@3 files to L3 => 433034151 bytes 2024/11/23-00:18:04.168019 140024886109760 (Original Log Time 2024/11/23-00:18:04.167949) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 12 0 0 0] max score 0.02, MB/sec: 12.1 rd, 11.5 wr, level 3, files in(1, 13) out(12 +0 blob) MB in(27.1, 409.2 +0.0 blob) out(413.0 +0.0 blob), read-write-amplify(31.3) write-amplify(15.2) OK, records in: 10827003, records dropped: 78526 output_compression: Snappy 2024/11/23-00:18:04.168026 140024886109760 (Original Log Time 2024/11/23-00:18:04.167995) EVENT_LOG_v1 {"time_micros": 1732292284167973, "cf": "default", "job": 15004, "event": "compaction_finished", "compaction_time_micros": 37740424, "compaction_time_cpu_micros": 17129350, "output_level": 3, "num_output_files": 12, "total_output_size": 433034151, "num_input_records": 10827003, "num_output_records": 10748477, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 271720089, "file_range_sync_nanos": 0, "file_fsync_nanos": 11931, "file_prepare_write_nanos": 29422, "lsm_state": [0, 0, 0, 12, 0, 0, 0]} 2024/11/23-00:18:04.298936 140024886109760 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732292246,"executesMs":37948,"compactionJobId":15004,"attempt":0,"compactionInputRawBytes":1566174262,"compactionInputZipBytes":457512940,"compactionOutputRawBytes":0,"compactionOutputZipBytes":433034151,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-92a1ce2cc08fcea4e2f320e8ca5961a1-2ac57eada49f1fb4-00"}}, response = 2024/11/23-00:18:04.299010 140024886109760 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-15004/att-00: olev 3, work 37.948 s, result 6.299 ms, install 0.709 ms, input{raw 1.459 GiB zip 436.318 MiB}