2024/11/23-19:14:45.908747 139957466945088 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00: fork to child time = 0.019801 sec 2024/11/23-19:14:45.909623 139957466945088 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00 2024/11/23-19:14:45.917644 139957466945088 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00 2024/11/23-19:14:45.918811 139957466945088 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-19:14:45.919173 139957466945088 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-19:14:45.971309 139957466945088 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-33980/att-00/MANIFEST-206347 2024/11/23-19:14:45.997897 139957466945088 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-33980/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 276717, last_sequence is 4209005248, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 276548 2024/11/23-19:14:45.997906 139957466945088 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-19:14:45.998046 139957466945088 [version_set.cc:5721] Creating manifest 276717 2024/11/23-19:14:46.033001 139957466945088 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-19:14:46.056784 139957466945088 [compaction_job.cc:2437] [default] [JOB 33980] Compacting 1@2 + 15@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-19:14:46.056804 139957466945088 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 25438 Base level 2, inputs: [276683(24MB)], [276589(37MB) 276633(37MB) 276635(38MB) 276636(37MB) 276637(36MB) 276638(38MB) 276639(35MB) 276640(34MB) 276641(37MB) 276642(34MB) 276643(37MB) 276644(35MB) 276645(35MB) 276646(36MB) 276708(37MB)] 2024/11/23-19:14:46.064297 139957466945088 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-19:14:46.064321 139957466945088 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-19:14:46.064336 139957466945088 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-19:14:46.064341 139957466945088 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-19:14:46.064350 139957466945088 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-19:14:46.658472 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276718.sst) /dev/shm/ToplingTemp/Topling-162634-47xi2O 2024/11/23-19:14:47.216652 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276718.sst): first pass time = 0.56's, 240.474'MB/sec 2024/11/23-19:14:48.883677 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276718.sst): old prealloc_size = 147639500, real_size = 35056544 2024/11/23-19:14:48.884411 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276718.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930033 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0331 GB avg-val = 116.32 avg-zval = 35.54 2024/11/23-19:14:48.885270 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276718.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:14:49.999532 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276718.sst): second pass time = 1.67's, 64.958'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.12's, 34.993'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2473.930'MB/sec (index lex order gen) rebuild zvType time = 0.98's, 0.237'MB/sec write SST data time = 0.13's, 290.644'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 36581.835'MB/sec zip my value time = 1.67's, unzip length = 0.108'GB zip my value throughput = 64.958'MB/sec zip pipeline throughput = 64.958'MB/sec entries = 930033 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.32 avg-zval = 35.54 usrkeys = 930033 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.32 avg-zval = 35.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930033 bytes = 3138880 } seq expand size = 6510231 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0331 GB dict = 1.08 MB all = 0.0390 GB } UnZip/Zip{ index = 10.5135 value = 3.2727 dict = 1.00 all = 3.4373 } Zip/UnZip{ index = 0.0951 value = 0.3056 dict = 1.00 all = 0.2909 } ---------------------------- total value len = 8.963307 GB avg = 0.168 KB (by entry num) total key len = 1.466862 GB avg = 0.028 KB total ukey len = 1.016479 GB avg = 0.015 KB total ukey num = 0.066480468 Billion total entry num = 0.053288197 Billion write speed all = 0.280548421 MB/sec (with seq num) write speed all = 0.269081746 MB/sec (without seq num) 2024/11/23-19:14:50.047932 139957466945088 [compaction_job.cc:1948] [default] [JOB 33980] Generated table #276718: 930033 keys, 39047372 bytes, temperature: kWarm 2024/11/23-19:14:50.048046 139957466945088 EVENT_LOG_v1 {"time_micros": 1732360490047969, "cf_name": "default", "job": 33980, "event": "table_file_creation", "file_number": 276718, "file_size": 39047372, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4136224572, "largest_seqno": 4208709513, "table_properties": {"data_size": 33054608, "index_size": 1769216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26040924, "raw_average_key_size": 28, "raw_value_size": 108176826, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930033, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360486, "oldest_key_time": 0, "file_creation_time": 1732360486, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276718, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:14:50.086433 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276719.sst) /dev/shm/ToplingTemp/Topling-162634-dD484O 2024/11/23-19:14:51.083148 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276719.sst): first pass time = 1.00's, 134.665'MB/sec 2024/11/23-19:14:52.728395 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276719.sst): old prealloc_size = 147639500, real_size = 34979712 2024/11/23-19:14:52.729173 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276719.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926774 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0330 GB avg-val = 116.82 avg-zval = 35.59 2024/11/23-19:14:52.729969 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276719.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:14:53.805354 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276719.sst): second pass time = 1.64's, 65.873'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.08's, 36.071'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2325.559'MB/sec (index lex order gen) rebuild zvType time = 0.89's, 0.261'MB/sec write SST data time = 0.19's, 207.335'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 39636.651'MB/sec zip my value time = 1.64's, unzip length = 0.108'GB zip my value throughput = 65.873'MB/sec zip pipeline throughput = 65.873'MB/sec entries = 926774 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.82 avg-zval = 35.59 usrkeys = 926774 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.82 avg-zval = 35.59 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926774 bytes = 3012032 } seq expand size = 6487418 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0330 GB dict = 1.08 MB all = 0.0388 GB } UnZip/Zip{ index = 10.5113 value = 3.2824 dict = 1.00 all = 3.4551 } Zip/UnZip{ index = 0.0951 value = 0.3047 dict = 1.00 all = 0.2894 } ---------------------------- total value len = 9.078062 GB avg = 0.167 KB (by entry num) total key len = 1.492812 GB avg = 0.028 KB total ukey len = 1.035014 GB avg = 0.015 KB total ukey num = 0.067407242 Billion total entry num = 0.054214971 Billion write speed all = 0.284303976 MB/sec (with seq num) write speed all = 0.272639070 MB/sec (without seq num) 2024/11/23-19:14:53.861166 139957466945088 [compaction_job.cc:1948] [default] [JOB 33980] Generated table #276719: 926774 keys, 38846082 bytes, temperature: kWarm 2024/11/23-19:14:53.861236 139957466945088 EVENT_LOG_v1 {"time_micros": 1732360493861200, "cf_name": "default", "job": 33980, "event": "table_file_creation", "file_number": 276719, "file_size": 38846082, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4155809461, "largest_seqno": 4208709565, "table_properties": {"data_size": 32984432, "index_size": 1763392, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25949672, "raw_average_key_size": 28, "raw_value_size": 108268116, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926774, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360490, "oldest_key_time": 0, "file_creation_time": 1732360490, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276719, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:14:53.900954 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276720.sst) /dev/shm/ToplingTemp/Topling-162634-CCgWeP 2024/11/23-19:14:55.125952 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276720.sst): first pass time = 1.22's, 109.571'MB/sec 2024/11/23-19:14:56.749195 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276720.sst): old prealloc_size = 147639500, real_size = 36784576 2024/11/23-19:14:56.749897 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276720.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 936586 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1080 GB zip-val = 0.0348 GB avg-val = 115.31 avg-zval = 37.13 2024/11/23-19:14:56.750807 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276720.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:14:57.700744 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276720.sst): second pass time = 1.62's, 66.568'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 43.245'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2612.854'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.292'MB/sec write SST data time = 0.15's, 275.595'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 36278.437'MB/sec zip my value time = 1.62's, unzip length = 0.108'GB zip my value throughput = 66.568'MB/sec zip pipeline throughput = 66.568'MB/sec entries = 936586 avg-key = 28.00 avg-zkey = 1.89 avg-val = 115.31 avg-zval = 37.13 usrkeys = 936586 avg-key = 20.00 avg-zkey = 1.89 avg-val = 122.31 avg-zval = 37.13 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 936586 bytes = 3512208 } seq expand size = 6556102 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0348 GB dict = 1.08 MB all = 0.0411 GB } UnZip/Zip{ index = 10.5815 value = 3.1050 dict = 1.00 all = 3.2617 } Zip/UnZip{ index = 0.0945 value = 0.3221 dict = 1.00 all = 0.3066 } ---------------------------- total value len = 9.192612 GB avg = 0.167 KB (by entry num) total key len = 1.519036 GB avg = 0.028 KB total ukey len = 1.053746 GB avg = 0.015 KB total ukey num = 0.068343828 Billion total entry num = 0.055151557 Billion write speed all = 0.288059915 MB/sec (with seq num) write speed all = 0.276194736 MB/sec (without seq num) 2024/11/23-19:14:57.757400 139957466945088 [compaction_job.cc:1948] [default] [JOB 33980] Generated table #276720: 936586 keys, 41149016 bytes, temperature: kWarm 2024/11/23-19:14:57.757470 139957466945088 EVENT_LOG_v1 {"time_micros": 1732360497757434, "cf_name": "default", "job": 33980, "event": "table_file_creation", "file_number": 276720, "file_size": 41149016, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3591110068, "largest_seqno": 4208709697, "table_properties": {"data_size": 34780016, "index_size": 1770240, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26224408, "raw_average_key_size": 28, "raw_value_size": 107993384, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 936586, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360493, "oldest_key_time": 0, "file_creation_time": 1732360493, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276720, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:14:57.796001 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276721.sst) /dev/shm/ToplingTemp/Topling-162634-hjY8pN 2024/11/23-19:14:59.004417 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276721.sst): first pass time = 1.21's, 111.073'MB/sec 2024/11/23-19:15:00.517445 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276721.sst): old prealloc_size = 147639500, real_size = 36612000 2024/11/23-19:15:00.518041 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276721.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929751 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0346 GB avg-val = 116.36 avg-zval = 37.20 2024/11/23-19:15:00.518677 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276721.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:15:01.322849 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276721.sst): second pass time = 1.51's, 71.538'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 49.604'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3103.406'MB/sec (index lex order gen) rebuild zvType time = 0.67's, 0.346'MB/sec write SST data time = 0.13's, 300.719'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 31973.808'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.538'MB/sec zip pipeline throughput = 71.538'MB/sec entries = 929751 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.36 avg-zval = 37.20 usrkeys = 929751 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.05 avg-zval = 37.20 TagRS{ kinds = RS_Key0_TagN bytes = 145360 } TagArray{ size = 622959 bytes = 2336112 } seq expand size = 4360713 multi value expand size = 0 cnt WriteAppend = 23 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0346 GB dict = 1.08 MB all = 0.0399 GB } UnZip/Zip{ index = 10.3756 value = 3.1279 dict = 1.00 all = 3.3597 } Zip/UnZip{ index = 0.0964 value = 0.3197 dict = 1.00 all = 0.2976 } ---------------------------- total value len = 9.305157 GB avg = 0.166 KB (by entry num) total key len = 1.545069 GB avg = 0.028 KB total ukey len = 1.072341 GB avg = 0.015 KB total ukey num = 0.069273579 Billion total entry num = 0.056081308 Billion write speed all = 0.291758180 MB/sec (with seq num) write speed all = 0.279694151 MB/sec (without seq num) 2024/11/23-19:15:01.374848 139957466945088 [compaction_job.cc:1948] [default] [JOB 33980] Generated table #276721: 929751 keys, 39949756 bytes, temperature: kWarm 2024/11/23-19:15:01.374919 139957466945088 EVENT_LOG_v1 {"time_micros": 1732360501374883, "cf_name": "default", "job": 33980, "event": "table_file_creation", "file_number": 276721, "file_size": 39949756, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012775967, "table_properties": {"data_size": 34587216, "index_size": 1792192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26033028, "raw_average_key_size": 28, "raw_value_size": 108184826, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929751, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360497, "oldest_key_time": 0, "file_creation_time": 1732360497, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276721, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:15:01.413658 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276722.sst) /dev/shm/ToplingTemp/Topling-162634-ITCLoO 2024/11/23-19:15:02.436240 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276722.sst): first pass time = 1.02's, 131.260'MB/sec 2024/11/23-19:15:03.763740 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276722.sst): old prealloc_size = 147639500, real_size = 35141968 2024/11/23-19:15:03.764366 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276722.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925453 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0331 GB avg-val = 117.03 avg-zval = 35.82 2024/11/23-19:15:03.765163 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276722.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:15:04.607932 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276722.sst): second pass time = 1.33's, 81.632'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.616'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2915.730'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.13's, 300.102'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32466.771'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.632'MB/sec zip pipeline throughput = 81.632'MB/sec entries = 925453 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.03 avg-zval = 35.82 usrkeys = 925453 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.03 avg-zval = 35.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925453 bytes = 3354784 } seq expand size = 6478171 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0331 GB dict = 1.08 MB all = 0.0394 GB } UnZip/Zip{ index = 10.5138 value = 3.2671 dict = 1.00 all = 3.4107 } Zip/UnZip{ index = 0.0951 value = 0.3061 dict = 1.00 all = 0.2932 } ---------------------------- total value len = 9.419941 GB avg = 0.165 KB (by entry num) total key len = 1.570982 GB avg = 0.028 KB total ukey len = 1.090850 GB avg = 0.016 KB total ukey num = 0.070199032 Billion total entry num = 0.057006761 Billion write speed all = 0.295515335 MB/sec (with seq num) write speed all = 0.283253309 MB/sec (without seq num) 2024/11/23-19:15:04.657347 139957466945088 [compaction_job.cc:1948] [default] [JOB 33980] Generated table #276722: 925453 keys, 39351808 bytes, temperature: kWarm 2024/11/23-19:15:04.657421 139957466945088 EVENT_LOG_v1 {"time_micros": 1732360504657383, "cf_name": "default", "job": 33980, "event": "table_file_creation", "file_number": 276722, "file_size": 39351808, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012776021, "table_properties": {"data_size": 33149952, "index_size": 1760448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25912684, "raw_average_key_size": 28, "raw_value_size": 108305174, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925453, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360501, "oldest_key_time": 0, "file_creation_time": 1732360501, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276722, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:15:04.694875 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276723.sst) /dev/shm/ToplingTemp/Topling-162634-DpXjQN 2024/11/23-19:15:05.727024 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276723.sst): first pass time = 1.03's, 130.043'MB/sec 2024/11/23-19:15:07.091538 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276723.sst): old prealloc_size = 147639500, real_size = 36011360 2024/11/23-19:15:07.092133 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276723.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929041 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0340 GB avg-val = 116.47 avg-zval = 36.60 2024/11/23-19:15:07.092863 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276723.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:15:07.903037 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276723.sst): second pass time = 1.36's, 79.351'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 49.059'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3107.420'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.343'MB/sec write SST data time = 0.13's, 299.769'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32172.650'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.351'MB/sec zip pipeline throughput = 79.351'MB/sec entries = 929041 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.47 avg-zval = 36.60 usrkeys = 929041 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.09 avg-zval = 36.60 TagRS{ kinds = RS_Key0_TagN bytes = 145280 } TagArray{ size = 746044 bytes = 2797680 } seq expand size = 5222308 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0340 GB dict = 1.08 MB all = 0.0398 GB } UnZip/Zip{ index = 10.4471 value = 3.1825 dict = 1.00 all = 3.3714 } Zip/UnZip{ index = 0.0957 value = 0.3142 dict = 1.00 all = 0.2966 } ---------------------------- total value len = 9.533368 GB avg = 0.165 KB (by entry num) total key len = 1.596995 GB avg = 0.028 KB total ukey len = 1.109431 GB avg = 0.016 KB total ukey num = 0.071128073 Billion total entry num = 0.057935802 Billion write speed all = 0.299237980 MB/sec (with seq num) write speed all = 0.286777224 MB/sec (without seq num) 2024/11/23-19:15:07.953456 139957466945088 [compaction_job.cc:1948] [default] [JOB 33980] Generated table #276723: 929041 keys, 39810606 bytes, temperature: kWarm 2024/11/23-19:15:07.953532 139957466945088 EVENT_LOG_v1 {"time_micros": 1732360507953493, "cf_name": "default", "job": 33980, "event": "table_file_creation", "file_number": 276723, "file_size": 39810606, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012774830, "table_properties": {"data_size": 34000336, "index_size": 1778560, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26013148, "raw_average_key_size": 28, "raw_value_size": 108204662, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929041, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360504, "oldest_key_time": 0, "file_creation_time": 1732360504, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276723, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:15:07.996575 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276724.sst) /dev/shm/ToplingTemp/Topling-162634-3agicR 2024/11/23-19:15:09.025420 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276724.sst): first pass time = 1.03's, 130.459'MB/sec 2024/11/23-19:15:10.356158 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276724.sst): old prealloc_size = 147639500, real_size = 33131696 2024/11/23-19:15:10.356709 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276724.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928905 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.49 avg-zval = 33.52 2024/11/23-19:15:10.357544 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276724.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:15:11.219730 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276724.sst): second pass time = 1.33's, 81.369'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.254'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3342.714'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.316'MB/sec write SST data time = 0.13's, 288.990'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32300.278'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.369'MB/sec zip pipeline throughput = 81.369'MB/sec entries = 928905 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.49 avg-zval = 33.52 usrkeys = 928905 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.49 avg-zval = 33.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928905 bytes = 3367296 } seq expand size = 6502335 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5144 value = 3.4758 dict = 1.00 all = 3.5933 } Zip/UnZip{ index = 0.0951 value = 0.2877 dict = 1.00 all = 0.2783 } ---------------------------- total value len = 9.648079 GB avg = 0.164 KB (by entry num) total key len = 1.623004 GB avg = 0.028 KB total ukey len = 1.128009 GB avg = 0.016 KB total ukey num = 0.072056978 Billion total entry num = 0.058864707 Billion write speed all = 0.302994201 MB/sec (with seq num) write speed all = 0.290334786 MB/sec (without seq num) 2024/11/23-19:15:11.265559 139957466945088 [compaction_job.cc:1948] [default] [JOB 33980] Generated table #276724: 928905 keys, 37352422 bytes, temperature: kWarm 2024/11/23-19:15:11.265630 139957466945088 EVENT_LOG_v1 {"time_micros": 1732360511265593, "cf_name": "default", "job": 33980, "event": "table_file_creation", "file_number": 276724, "file_size": 37352422, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776210, "table_properties": {"data_size": 31132384, "index_size": 1766912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26009340, "raw_average_key_size": 28, "raw_value_size": 108208490, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928905, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360507, "oldest_key_time": 0, "file_creation_time": 1732360507, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276724, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:15:11.301292 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276725.sst) /dev/shm/ToplingTemp/Topling-162634-NPZdhO 2024/11/23-19:15:12.319024 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276725.sst): first pass time = 1.02's, 131.884'MB/sec 2024/11/23-19:15:13.634693 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276725.sst): old prealloc_size = 147639500, real_size = 32226544 2024/11/23-19:15:13.635315 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276725.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 935875 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0302 GB avg-val = 115.41 avg-zval = 32.28 2024/11/23-19:15:13.636120 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276725.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:15:14.501158 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276725.sst): second pass time = 1.31's, 82.149'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.090'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2953.420'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.317'MB/sec write SST data time = 0.13's, 284.267'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 31805.699'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.149'MB/sec zip pipeline throughput = 82.149'MB/sec entries = 935875 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.41 avg-zval = 32.28 usrkeys = 935875 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.41 avg-zval = 32.28 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 935875 bytes = 3392560 } seq expand size = 6551125 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5119 value = 3.5752 dict = 1.00 all = 3.6804 } Zip/UnZip{ index = 0.0951 value = 0.2797 dict = 1.00 all = 0.2717 } ---------------------------- total value len = 9.762643 GB avg = 0.163 KB (by entry num) total key len = 1.649209 GB avg = 0.028 KB total ukey len = 1.146726 GB avg = 0.016 KB total ukey num = 0.072992853 Billion total entry num = 0.059800582 Billion write speed all = 0.306751354 MB/sec (with seq num) write speed all = 0.293891805 MB/sec (without seq num) 2024/11/23-19:15:14.546124 139957466945088 [compaction_job.cc:1948] [default] [JOB 33980] Generated table #276725: 935875 keys, 36468156 bytes, temperature: kWarm 2024/11/23-19:15:14.546195 139957466945088 EVENT_LOG_v1 {"time_micros": 1732360514546158, "cf_name": "default", "job": 33980, "event": "table_file_creation", "file_number": 276725, "file_size": 36468156, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776248, "table_properties": {"data_size": 30211808, "index_size": 1780608, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26204500, "raw_average_key_size": 28, "raw_value_size": 108013318, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 935875, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360511, "oldest_key_time": 0, "file_creation_time": 1732360511, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276725, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:15:14.589077 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276726.sst) /dev/shm/ToplingTemp/Topling-162634-J8KpmN 2024/11/23-19:15:15.545019 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276726.sst): first pass time = 0.96's, 140.410'MB/sec 2024/11/23-19:15:16.884752 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276726.sst): old prealloc_size = 147639500, real_size = 35155632 2024/11/23-19:15:16.885353 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276726.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929957 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0331 GB avg-val = 116.33 avg-zval = 35.64 2024/11/23-19:15:16.886172 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276726.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:15:17.723267 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276726.sst): second pass time = 1.34's, 80.822'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 47.103'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3054.797'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 312.668'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 31147.774'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.822'MB/sec zip pipeline throughput = 80.822'MB/sec entries = 929957 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.33 avg-zval = 35.64 usrkeys = 929957 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.33 avg-zval = 35.64 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929957 bytes = 3487360 } seq expand size = 6509699 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0331 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.4439 value = 3.2641 dict = 1.00 all = 3.3983 } Zip/UnZip{ index = 0.0957 value = 0.3064 dict = 1.00 all = 0.2943 } ---------------------------- total value len = 9.877332 GB avg = 0.163 KB (by entry num) total key len = 1.675248 GB avg = 0.028 KB total ukey len = 1.165326 GB avg = 0.016 KB total ukey num = 0.073922810 Billion total entry num = 0.060730539 Billion write speed all = 0.310507225 MB/sec (with seq num) write speed all = 0.297448828 MB/sec (without seq num) 2024/11/23-19:15:17.770519 139957466945088 [compaction_job.cc:1948] [default] [JOB 33980] Generated table #276726: 929957 keys, 39495172 bytes, temperature: kWarm 2024/11/23-19:15:17.770590 139957466945088 EVENT_LOG_v1 {"time_micros": 1732360517770553, "cf_name": "default", "job": 33980, "event": "table_file_creation", "file_number": 276726, "file_size": 39495172, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776268, "table_properties": {"data_size": 33142112, "index_size": 1780864, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26038796, "raw_average_key_size": 28, "raw_value_size": 108178970, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929957, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360514, "oldest_key_time": 0, "file_creation_time": 1732360514, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276726, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:15:17.812315 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276727.sst) /dev/shm/ToplingTemp/Topling-162634-IAL1hN 2024/11/23-19:15:18.686800 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276727.sst): first pass time = 0.87's, 153.488'MB/sec 2024/11/23-19:15:19.980310 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276727.sst): old prealloc_size = 147639500, real_size = 30654288 2024/11/23-19:15:19.980846 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276727.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931242 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0286 GB avg-val = 116.13 avg-zval = 30.75 2024/11/23-19:15:19.983556 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276727.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:15:20.809255 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276727.sst): second pass time = 1.29's, 83.696'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.080'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3476.362'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.333'MB/sec write SST data time = 0.13's, 270.878'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34035.347'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.696'MB/sec zip pipeline throughput = 83.696'MB/sec entries = 931242 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.13 avg-zval = 30.75 usrkeys = 931242 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.13 avg-zval = 30.75 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931242 bytes = 3375760 } seq expand size = 6518694 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0286 GB dict = 1.08 MB all = 0.0349 GB } UnZip/Zip{ index = 10.4437 value = 3.7762 dict = 1.00 all = 3.8478 } Zip/UnZip{ index = 0.0958 value = 0.2648 dict = 1.00 all = 0.2599 } ---------------------------- total value len = 9.991993 GB avg = 0.162 KB (by entry num) total key len = 1.701322 GB avg = 0.028 KB total ukey len = 1.183950 GB avg = 0.016 KB total ukey num = 0.074854052 Billion total entry num = 0.061661781 Billion write speed all = 0.314263836 MB/sec (with seq num) write speed all = 0.301006301 MB/sec (without seq num) 2024/11/23-19:15:20.853950 139957466945088 [compaction_job.cc:1948] [default] [JOB 33980] Generated table #276727: 931242 keys, 34881532 bytes, temperature: kWarm 2024/11/23-19:15:20.854046 139957466945088 EVENT_LOG_v1 {"time_micros": 1732360520854005, "cf_name": "default", "job": 33980, "event": "table_file_creation", "file_number": 276727, "file_size": 34881532, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776300, "table_properties": {"data_size": 28637952, "index_size": 1783360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26074776, "raw_average_key_size": 28, "raw_value_size": 108142972, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931242, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360517, "oldest_key_time": 0, "file_creation_time": 1732360517, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276727, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:15:20.890562 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276728.sst) /dev/shm/ToplingTemp/Topling-162634-hEkwjO 2024/11/23-19:15:21.791258 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276728.sst): first pass time = 0.90's, 149.021'MB/sec 2024/11/23-19:15:23.118511 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276728.sst): old prealloc_size = 147639500, real_size = 35291744 2024/11/23-19:15:23.119065 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276728.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922697 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0333 GB avg-val = 117.46 avg-zval = 36.10 2024/11/23-19:15:23.121194 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276728.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:15:23.960414 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276728.sst): second pass time = 1.33's, 81.703'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 47.050'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3277.382'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.323'MB/sec write SST data time = 0.13's, 310.512'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 36477.637'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.703'MB/sec zip pipeline throughput = 81.703'MB/sec entries = 922697 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.46 avg-zval = 36.10 usrkeys = 922697 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.46 avg-zval = 36.10 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922697 bytes = 3460128 } seq expand size = 6458879 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0333 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.5135 value = 3.2542 dict = 1.00 all = 3.3885 } Zip/UnZip{ index = 0.0951 value = 0.3073 dict = 1.00 all = 0.2951 } ---------------------------- total value len = 10.106834 GB avg = 0.161 KB (by entry num) total key len = 1.727158 GB avg = 0.028 KB total ukey len = 1.202404 GB avg = 0.016 KB total ukey num = 0.075776749 Billion total entry num = 0.062584478 Billion write speed all = 0.318017661 MB/sec (with seq num) write speed all = 0.304562882 MB/sec (without seq num) 2024/11/23-19:15:24.008739 139957466945088 [compaction_job.cc:1948] [default] [JOB 33980] Generated table #276728: 922697 keys, 39610086 bytes, temperature: kWarm 2024/11/23-19:15:24.008812 139957466945088 EVENT_LOG_v1 {"time_micros": 1732360524008775, "cf_name": "default", "job": 33980, "event": "table_file_creation", "file_number": 276728, "file_size": 39610086, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776316, "table_properties": {"data_size": 33305616, "index_size": 1755264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25835516, "raw_average_key_size": 28, "raw_value_size": 108382282, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922697, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360520, "oldest_key_time": 0, "file_creation_time": 1732360520, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276728, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:15:24.047946 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276729.sst) /dev/shm/ToplingTemp/Topling-162634-LWq6yP 2024/11/23-19:15:25.060005 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276729.sst): first pass time = 1.01's, 132.625'MB/sec 2024/11/23-19:15:26.394867 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276729.sst): old prealloc_size = 147639500, real_size = 32374512 2024/11/23-19:15:26.395487 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276729.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931083 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0304 GB avg-val = 116.15 avg-zval = 32.61 2024/11/23-19:15:26.397579 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276729.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:15:27.246749 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276729.sst): second pass time = 1.33's, 81.066'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.103'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2978.213'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 285.237'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33121.533'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.066'MB/sec zip pipeline throughput = 81.066'MB/sec entries = 931083 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.15 avg-zval = 32.61 usrkeys = 931083 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.15 avg-zval = 32.61 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931083 bytes = 3491584 } seq expand size = 6517581 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.4441 value = 3.5623 dict = 1.00 all = 3.6554 } Zip/UnZip{ index = 0.0957 value = 0.2807 dict = 1.00 all = 0.2736 } ---------------------------- total value len = 10.221500 GB avg = 0.161 KB (by entry num) total key len = 1.753228 GB avg = 0.028 KB total ukey len = 1.221026 GB avg = 0.016 KB total ukey num = 0.076707832 Billion total entry num = 0.063515561 Billion write speed all = 0.321771260 MB/sec (with seq num) write speed all = 0.308117517 MB/sec (without seq num) 2024/11/23-19:15:27.292744 139957466945088 [compaction_job.cc:1948] [default] [JOB 33980] Generated table #276729: 931083 keys, 36717652 bytes, temperature: kWarm 2024/11/23-19:15:27.292816 139957466945088 EVENT_LOG_v1 {"time_micros": 1732360527292776, "cf_name": "default", "job": 33980, "event": "table_file_creation", "file_number": 276729, "file_size": 36717652, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012776344, "table_properties": {"data_size": 30358560, "index_size": 1782976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26070324, "raw_average_key_size": 28, "raw_value_size": 108147522, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931083, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360524, "oldest_key_time": 0, "file_creation_time": 1732360524, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276729, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:15:27.330513 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276730.sst) /dev/shm/ToplingTemp/Topling-162634-dYSDFQ 2024/11/23-19:15:28.363846 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276730.sst): first pass time = 1.03's, 129.893'MB/sec 2024/11/23-19:15:29.688841 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276730.sst): old prealloc_size = 147639500, real_size = 32514592 2024/11/23-19:15:29.689420 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276730.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926230 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0305 GB avg-val = 116.91 avg-zval = 32.94 2024/11/23-19:15:29.690825 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276730.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:15:30.511834 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276730.sst): second pass time = 1.32's, 81.786'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 44.122'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3179.888'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 277.744'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 30759.907'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.786'MB/sec zip pipeline throughput = 81.786'MB/sec entries = 926230 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.91 avg-zval = 32.94 usrkeys = 926230 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.73 avg-zval = 32.94 TagRS{ kinds = RS_Key0_TagN bytes = 144840 } TagArray{ size = 770233 bytes = 2792112 } seq expand size = 5391631 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.4433 value = 3.5492 dict = 1.00 all = 3.6963 } Zip/UnZip{ index = 0.0958 value = 0.2818 dict = 1.00 all = 0.2705 } ---------------------------- total value len = 10.335175 GB avg = 0.160 KB (by entry num) total key len = 1.779163 GB avg = 0.028 KB total ukey len = 1.239551 GB avg = 0.016 KB total ukey num = 0.077634062 Billion total entry num = 0.064441791 Billion write speed all = 0.325494129 MB/sec (with seq num) write speed all = 0.311642492 MB/sec (without seq num) 2024/11/23-19:15:30.558070 139957466945088 [compaction_job.cc:1948] [default] [JOB 33980] Generated table #276730: 926230 keys, 36311014 bytes, temperature: kWarm 2024/11/23-19:15:30.558149 139957466945088 EVENT_LOG_v1 {"time_micros": 1732360530558111, "cf_name": "default", "job": 33980, "event": "table_file_creation", "file_number": 276730, "file_size": 36311014, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 30509008, "index_size": 1773824, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25934440, "raw_average_key_size": 28, "raw_value_size": 108283364, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926230, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360527, "oldest_key_time": 0, "file_creation_time": 1732360527, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276730, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:15:30.595145 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276731.sst) /dev/shm/ToplingTemp/Topling-162634-CKeggN 2024/11/23-19:15:31.604005 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276731.sst): first pass time = 1.01's, 133.044'MB/sec 2024/11/23-19:15:33.095487 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276731.sst): old prealloc_size = 147639500, real_size = 36802064 2024/11/23-19:15:33.096067 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276731.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926673 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0348 GB avg-val = 116.84 avg-zval = 37.54 2024/11/23-19:15:33.096318 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276731.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:15:33.809171 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276731.sst): second pass time = 1.49's, 72.635'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.71's, 53.650'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3154.607'MB/sec (index lex order gen) rebuild zvType time = 0.59's, 0.393'MB/sec write SST data time = 0.12's, 310.131'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33586.833'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.635'MB/sec zip pipeline throughput = 72.635'MB/sec entries = 926673 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.84 avg-zval = 37.54 usrkeys = 926673 avg-key = 20.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 37.54 TagRS{ kinds = RS_Key0_TagN bytes = 144880 } TagArray{ size = 154398 bytes = 482512 } seq expand size = 1080786 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0348 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.3772 value = 3.1126 dict = 1.00 all = 3.5055 } Zip/UnZip{ index = 0.0964 value = 0.3213 dict = 1.00 all = 0.2853 } ---------------------------- total value len = 10.444526 GB avg = 0.160 KB (by entry num) total key len = 1.805109 GB avg = 0.028 KB total ukey len = 1.258084 GB avg = 0.016 KB total ukey num = 0.078560735 Billion total entry num = 0.065368464 Billion write speed all = 0.329100242 MB/sec (with seq num) write speed all = 0.315050663 MB/sec (without seq num) 2024/11/23-19:15:33.856350 139957466945088 [compaction_job.cc:1948] [default] [JOB 33980] Generated table #276731: 926673 keys, 38287833 bytes, temperature: kWarm 2024/11/23-19:15:33.856418 139957466945088 EVENT_LOG_v1 {"time_micros": 1732360533856384, "cf_name": "default", "job": 33980, "event": "table_file_creation", "file_number": 276731, "file_size": 38287833, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700812, "table_properties": {"data_size": 34784256, "index_size": 1785984, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25946844, "raw_average_key_size": 28, "raw_value_size": 108270894, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926673, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360530, "oldest_key_time": 0, "file_creation_time": 1732360530, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276731, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:15:33.896034 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276732.sst) /dev/shm/ToplingTemp/Topling-162634-l8nOgR 2024/11/23-19:15:34.924192 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276732.sst): first pass time = 1.03's, 130.546'MB/sec 2024/11/23-19:15:36.323290 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276732.sst): old prealloc_size = 147639500, real_size = 35714976 2024/11/23-19:15:36.323886 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276732.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925630 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0337 GB avg-val = 117.00 avg-zval = 36.44 2024/11/23-19:15:36.324475 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276732.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:15:37.108821 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276732.sst): second pass time = 1.40's, 77.451'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.79's, 49.206'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3046.764'MB/sec (index lex order gen) rebuild zvType time = 0.66's, 0.352'MB/sec write SST data time = 0.13's, 302.927'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32106.549'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.451'MB/sec zip pipeline throughput = 77.451'MB/sec entries = 925630 avg-key = 28.00 avg-zkey = 1.89 avg-val = 117.00 avg-zval = 36.44 usrkeys = 925630 avg-key = 20.00 avg-zkey = 1.89 avg-val = 121.18 avg-zval = 36.44 TagRS{ kinds = RS_Key0_TagN bytes = 144720 } TagArray{ size = 552309 bytes = 1933104 } seq expand size = 3866163 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0337 GB dict = 1.08 MB all = 0.0387 GB } UnZip/Zip{ index = 10.5828 value = 3.2104 dict = 1.00 all = 3.4725 } Zip/UnZip{ index = 0.0945 value = 0.3115 dict = 1.00 all = 0.2880 } ---------------------------- total value len = 10.556693 GB avg = 0.159 KB (by entry num) total key len = 1.831027 GB avg = 0.028 KB total ukey len = 1.276597 GB avg = 0.016 KB total ukey num = 0.079486365 Billion total entry num = 0.066294094 Billion write speed all = 0.332780523 MB/sec (with seq num) write speed all = 0.318533263 MB/sec (without seq num) 2024/11/23-19:15:37.157650 139957466945088 [compaction_job.cc:1948] [default] [JOB 33980] Generated table #276732: 925630 keys, 38651537 bytes, temperature: kWarm 2024/11/23-19:15:37.157723 139957466945088 EVENT_LOG_v1 {"time_micros": 1732360537157685, "cf_name": "default", "job": 33980, "event": "table_file_creation", "file_number": 276732, "file_size": 38651537, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4208709709, "table_properties": {"data_size": 33734096, "index_size": 1749312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25917640, "raw_average_key_size": 28, "raw_value_size": 108300200, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925630, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360533, "oldest_key_time": 0, "file_creation_time": 1732360533, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276732, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:15:37.193605 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276733.sst) /dev/shm/ToplingTemp/Topling-162634-qP0tDO 2024/11/23-19:15:37.308302 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276733.sst): first pass time = 0.11's, 224.928'MB/sec 2024/11/23-19:15:37.534912 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276733.sst): old prealloc_size = 147639500, real_size = 7403728 2024/11/23-19:15:37.535121 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276733.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 179735 prefix = 4 raw-key = 0.0029 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0208 GB zip-val = 0.0070 GB avg-val = 115.49 avg-zval = 39.03 2024/11/23-19:15:37.535307 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276733.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0004 GB, waited 0.000 sec, Key+Value = 0.026 GB 2024/11/23-19:15:37.692130 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00/276733.sst): second pass time = 0.23's, 91.915'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 51.786'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1840.110'MB/sec (index lex order gen) rebuild zvType time = 0.13's, 0.341'MB/sec write SST data time = 0.03's, 320.187'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.218'MB, throughput = 7403.073'MB/sec zip my value time = 0.23's, unzip length = 0.021'GB zip my value throughput = 91.915'MB/sec zip pipeline throughput = 91.915'MB/sec entries = 179735 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.49 avg-zval = 39.03 usrkeys = 179735 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.49 avg-zval = 39.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 179735 bytes = 561680 } seq expand size = 1258145 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0036 GB value = 0.0208 GB dict = 0.22 MB all = 0.0258 GB } __ZipSize{ index = 0.0003 GB value = 0.0070 GB dict = 0.22 MB all = 0.0081 GB } UnZip/Zip{ index = 10.4731 value = 2.9590 dict = 1.00 all = 3.1683 } Zip/UnZip{ index = 0.0955 value = 0.3380 dict = 1.00 all = 0.3156 } ---------------------------- total value len = 10.578709 GB avg = 0.159 KB (by entry num) total key len = 1.836060 GB avg = 0.028 KB total ukey len = 1.280191 GB avg = 0.016 KB total ukey num = 0.079666100 Billion total entry num = 0.066473829 Billion write speed all = 0.333501936 MB/sec (with seq num) write speed all = 0.319216273 MB/sec (without seq num) 2024/11/23-19:15:37.709011 139957466945088 [compaction_job.cc:1948] [default] [JOB 33980] Generated table #276733: 179735 keys, 8140329 bytes, temperature: kWarm 2024/11/23-19:15:37.709069 139957466945088 EVENT_LOG_v1 {"time_micros": 1732360537709039, "cf_name": "default", "job": 33980, "event": "table_file_creation", "file_number": 276733, "file_size": 8140329, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4190623292, "largest_seqno": 4208709726, "table_properties": {"data_size": 7015360, "index_size": 343232, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 5032580, "raw_average_key_size": 28, "raw_value_size": 20758330, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 179735, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360537, "oldest_key_time": 0, "file_creation_time": 1732360537, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276733, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:15:37.719735 139957466945088 [compaction_job.cc:703] job-33980: subcompact[0], size: 0.584070 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33980/att-00 [276718,276719,276720,276721,276722,276723,276724,276725,276726,276727,276728,276729,276730,276731,276732,276733] 2024/11/23-19:15:37.818300 139957466945088 (Original Log Time 2024/11/23-19:14:46.056857) EVENT_LOG_v1 {"time_micros": 1732360486056817, "job": 33980, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [276683], "files_L3": [276589, 276633, 276635, 276636, 276637, 276638, 276639, 276640, 276641, 276642, 276643, 276644, 276645, 276646, 276708], "score": 0.992162, "input_data_size": 602235093, "oldest_snapshot_seqno": 4209004950} 2024/11/23-19:15:37.818303 139957466945088 (Original Log Time 2024/11/23-19:15:37.816938) [compaction_job.cc:2022] [default] [JOB 33980] Compacted 1@2 + 15@3 files to L3 => 584070373 bytes 2024/11/23-19:15:37.818304 139957466945088 (Original Log Time 2024/11/23-19:15:37.818235) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 16 0 0 0] max score 0.02, MB/sec: 11.7 rd, 11.3 wr, level 3, files in(1, 15) out(16 +0 blob) MB in(24.8, 549.5 +0.0 blob) out(557.0 +0.0 blob), read-write-amplify(45.5) write-amplify(22.4) OK, records in: 14150294, records dropped: 34629 output_compression: Snappy 2024/11/23-19:15:37.818310 139957466945088 (Original Log Time 2024/11/23-19:15:37.818278) EVENT_LOG_v1 {"time_micros": 1732360537818259, "cf": "default", "job": 33980, "event": "compaction_finished", "compaction_time_micros": 51662882, "compaction_time_cpu_micros": 21925007, "output_level": 3, "num_output_files": 16, "total_output_size": 584070373, "num_input_records": 14150294, "num_output_records": 14115665, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 404831251, "file_range_sync_nanos": 0, "file_fsync_nanos": 12326, "file_prepare_write_nanos": 43845, "lsm_state": [0, 0, 0, 16, 0, 0, 0]} 2024/11/23-19:15:37.962848 139957466945088 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732360485,"executesMs":51900,"compactionJobId":33980,"attempt":0,"compactionInputRawBytes":2043975268,"compactionInputZipBytes":602235093,"compactionOutputRawBytes":0,"compactionOutputZipBytes":584070373,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-bd9901027a511c3b8d061572649d2aaa-7544a5428aa12f44-00"}}, response = 2024/11/23-19:15:37.962931 139957466945088 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-33980/att-00: olev 3, work 51.900 s, result 7.747 ms, install 1.389 ms, input{raw 1.904 GiB zip 574.336 MiB}