2024/11/23-04:15:58.876126 140025171461696 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00: fork to child time = 0.008963 sec 2024/11/23-04:15:58.876898 140025171461696 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00 2024/11/23-04:15:58.884928 140025171461696 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00 2024/11/23-04:15:58.886055 140025171461696 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-04:15:58.886301 140025171461696 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-04:15:58.934810 140025171461696 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-19094/att-00/MANIFEST-206347 2024/11/23-04:15:58.971384 140025171461696 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-19094/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 244296, last_sequence is 3780524042, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 244091 2024/11/23-04:15:58.971390 140025171461696 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-04:15:58.971491 140025171461696 [version_set.cc:5721] Creating manifest 244296 2024/11/23-04:15:59.013344 140025171461696 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-04:15:59.037720 140025171461696 [compaction_job.cc:2437] [default] [JOB 19094] Compacting 1@2 + 16@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-04:15:59.037733 140025171461696 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 14277 Base level 2, inputs: [244245(27MB)], [244293(35MB) 244294(8584KB) 244196(35MB) 244197(36MB) 244198(34MB) 244199(36MB) 244200(34MB) 244201(33MB) 244202(35MB) 244203(33MB) 244204(34MB) 244205(12MB) 244159(35MB) 244160(35MB) 244161(34MB) 244274(34MB)] 2024/11/23-04:15:59.045215 140025171461696 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-04:15:59.045244 140025171461696 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-04:15:59.045263 140025171461696 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-04:15:59.045267 140025171461696 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-04:15:59.045278 140025171461696 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-04:15:59.508178 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244297.sst) /dev/shm/ToplingTemp/Topling-405930-W0vI7G 2024/11/23-04:16:00.081865 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244297.sst): first pass time = 0.57's, 233.975'MB/sec 2024/11/23-04:16:01.641742 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244297.sst): old prealloc_size = 147639500, real_size = 33224864 2024/11/23-04:16:01.642537 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244297.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 936859 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1080 GB zip-val = 0.0312 GB avg-val = 115.26 avg-zval = 33.30 2024/11/23-04:16:01.643584 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244297.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:16:02.709728 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244297.sst): second pass time = 1.56's, 69.271'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.07's, 34.974'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2371.796'MB/sec (index lex order gen) rebuild zvType time = 0.92's, 0.255'MB/sec write SST data time = 0.15's, 249.912'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 34019.340'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.271'MB/sec zip pipeline throughput = 69.271'MB/sec entries = 936859 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.26 avg-zval = 33.30 usrkeys = 936859 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.26 avg-zval = 33.30 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 936859 bytes = 3279024 } seq expand size = 6558013 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4430 value = 3.4615 dict = 1.00 all = 3.5933 } Zip/UnZip{ index = 0.0958 value = 0.2889 dict = 1.00 all = 0.2783 } ---------------------------- total value len = 0.114544 GB avg = 0.122 KB (by entry num) total key len = 0.026232 GB avg = 0.028 KB total ukey len = 0.018737 GB avg = 0.020 KB total ukey num = 0.000936859 Billion total entry num = 0.000936859 Billion write speed all = 0.002016279 MB/sec (with seq num) write speed all = 0.001908933 MB/sec (without seq num) 2024/11/23-04:16:02.749197 140025171461696 [compaction_job.cc:1948] [default] [JOB 19094] Generated table #244297: 936859 keys, 37352078 bytes, temperature: kWarm 2024/11/23-04:16:02.749289 140025171461696 EVENT_LOG_v1 {"time_micros": 1732306562749234, "cf_name": "default", "job": 19094, "event": "table_file_creation", "file_number": 244297, "file_size": 37352078, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 3780163192, "table_properties": {"data_size": 31196240, "index_size": 1794240, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26232052, "raw_average_key_size": 28, "raw_value_size": 107985770, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 936859, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732306559, "oldest_key_time": 0, "file_creation_time": 1732306559, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 244297, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:16:02.788216 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244298.sst) /dev/shm/ToplingTemp/Topling-405930-8QhxOH 2024/11/23-04:16:03.897594 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244298.sst): first pass time = 1.11's, 120.990'MB/sec 2024/11/23-04:16:05.536933 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244298.sst): old prealloc_size = 147639500, real_size = 32795376 2024/11/23-04:16:05.537508 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244298.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928723 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0308 GB avg-val = 116.52 avg-zval = 33.16 2024/11/23-04:16:05.538285 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244298.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:16:06.453336 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244298.sst): second pass time = 1.64's, 66.038'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.393'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3188.771'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.294'MB/sec write SST data time = 0.13's, 295.148'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33488.681'MB/sec zip my value time = 1.64's, unzip length = 0.108'GB zip my value throughput = 66.038'MB/sec zip pipeline throughput = 86.774'MB/sec entries = 928723 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.52 avg-zval = 33.16 usrkeys = 928723 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.52 avg-zval = 33.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928723 bytes = 3366640 } seq expand size = 6501061 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5147 value = 3.5138 dict = 1.00 all = 3.6260 } Zip/UnZip{ index = 0.0951 value = 0.2846 dict = 1.00 all = 0.2758 } ---------------------------- total value len = 0.229258 GB avg = 0.123 KB (by entry num) total key len = 0.052236 GB avg = 0.028 KB total ukey len = 0.037312 GB avg = 0.020 KB total ukey num = 0.001865582 Billion total entry num = 0.001865582 Billion write speed all = 0.004031527 MB/sec (with seq num) write speed all = 0.003817778 MB/sec (without seq num) 2024/11/23-04:16:06.486158 140025171461696 [compaction_job.cc:1948] [default] [JOB 19094] Generated table #244298: 928723 keys, 37015572 bytes, temperature: kWarm 2024/11/23-04:16:06.486223 140025171461696 EVENT_LOG_v1 {"time_micros": 1732306566486190, "cf_name": "default", "job": 19094, "event": "table_file_creation", "file_number": 244298, "file_size": 37015572, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3780163203, "table_properties": {"data_size": 30796512, "index_size": 1766528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26004244, "raw_average_key_size": 28, "raw_value_size": 108213598, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928723, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732306562, "oldest_key_time": 0, "file_creation_time": 1732306562, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 244298, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:16:06.523099 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244299.sst) /dev/shm/ToplingTemp/Topling-405930-gfrhiI 2024/11/23-04:16:07.420731 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244299.sst): first pass time = 0.90's, 149.530'MB/sec 2024/11/23-04:16:09.218987 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244299.sst): old prealloc_size = 147639500, real_size = 33146304 2024/11/23-04:16:09.219573 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244299.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932495 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0311 GB avg-val = 115.93 avg-zval = 33.39 2024/11/23-04:16:09.220356 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244299.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:16:10.314024 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244299.sst): second pass time = 1.80's, 60.141'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.09's, 34.134'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3124.701'MB/sec (index lex order gen) rebuild zvType time = 0.86's, 0.270'MB/sec write SST data time = 0.23's, 161.709'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 30684.832'MB/sec zip my value time = 1.80's, unzip length = 0.108'GB zip my value throughput = 60.141'MB/sec zip pipeline throughput = 116.851'MB/sec entries = 932495 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.93 avg-zval = 33.39 usrkeys = 932495 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.93 avg-zval = 33.39 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932495 bytes = 3380304 } seq expand size = 6527465 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5155 value = 3.4717 dict = 1.00 all = 3.5909 } Zip/UnZip{ index = 0.0951 value = 0.2880 dict = 1.00 all = 0.2785 } ---------------------------- total value len = 0.343894 GB avg = 0.123 KB (by entry num) total key len = 0.078346 GB avg = 0.028 KB total ukey len = 0.055962 GB avg = 0.020 KB total ukey num = 0.002798077 Billion total entry num = 0.002798077 Billion write speed all = 0.006046925 MB/sec (with seq num) write speed all = 0.005726353 MB/sec (without seq num) 2024/11/23-04:16:10.350529 140025171461696 [compaction_job.cc:1948] [default] [JOB 19094] Generated table #244299: 932495 keys, 37377026 bytes, temperature: kWarm 2024/11/23-04:16:10.350611 140025171461696 EVENT_LOG_v1 {"time_micros": 1732306570350567, "cf_name": "default", "job": 19094, "event": "table_file_creation", "file_number": 244299, "file_size": 37377026, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3504428200, "largest_seqno": 3780163234, "table_properties": {"data_size": 31139440, "index_size": 1773568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26109860, "raw_average_key_size": 28, "raw_value_size": 108107874, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932495, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732306566, "oldest_key_time": 0, "file_creation_time": 1732306566, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 244299, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:16:10.413950 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244300.sst) /dev/shm/ToplingTemp/Topling-405930-IjKNPG 2024/11/23-04:16:11.219945 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244300.sst): first pass time = 0.81's, 166.535'MB/sec 2024/11/23-04:16:12.848143 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244300.sst): old prealloc_size = 147639500, real_size = 34853248 2024/11/23-04:16:12.848748 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244300.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929191 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0329 GB avg-val = 116.45 avg-zval = 35.36 2024/11/23-04:16:12.849251 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244300.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:16:13.658161 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244300.sst): second pass time = 1.63's, 66.484'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 46.340'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3021.672'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.340'MB/sec write SST data time = 0.13's, 299.070'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 28166.675'MB/sec zip my value time = 1.63's, unzip length = 0.108'GB zip my value throughput = 66.484'MB/sec zip pipeline throughput = 67.697'MB/sec entries = 929191 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.45 avg-zval = 35.36 usrkeys = 929191 avg-key = 20.00 avg-zkey = 1.90 avg-val = 120.06 avg-zval = 35.36 TagRS{ kinds = RS_Key0_TagN bytes = 145280 } TagArray{ size = 479955 bytes = 1679856 } seq expand size = 3359685 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0329 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5143 value = 3.2934 dict = 1.00 all = 3.5758 } Zip/UnZip{ index = 0.0951 value = 0.3036 dict = 1.00 all = 0.2797 } ---------------------------- total value len = 0.455454 GB avg = 0.122 KB (by entry num) total key len = 0.104364 GB avg = 0.028 KB total ukey len = 0.074545 GB avg = 0.020 KB total ukey num = 0.003727268 Billion total entry num = 0.003727268 Billion write speed all = 0.008016797 MB/sec (with seq num) write speed all = 0.007589790 MB/sec (without seq num) 2024/11/23-04:16:13.692028 140025171461696 [compaction_job.cc:1948] [default] [JOB 19094] Generated table #244300: 929191 keys, 37535147 bytes, temperature: kWarm 2024/11/23-04:16:13.692094 140025171461696 EVENT_LOG_v1 {"time_micros": 1732306573692060, "cf_name": "default", "job": 19094, "event": "table_file_creation", "file_number": 244300, "file_size": 37535147, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3780163246, "table_properties": {"data_size": 32853296, "index_size": 1767488, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26017348, "raw_average_key_size": 28, "raw_value_size": 108200458, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929191, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732306570, "oldest_key_time": 0, "file_creation_time": 1732306570, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 244300, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:16:13.735822 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244301.sst) /dev/shm/ToplingTemp/Topling-405930-QARFwJ 2024/11/23-04:16:14.623904 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244301.sst): first pass time = 0.89's, 151.139'MB/sec 2024/11/23-04:16:15.995017 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244301.sst): old prealloc_size = 147639500, real_size = 35535920 2024/11/23-04:16:15.995663 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244301.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1080418 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1040 GB zip-val = 0.0332 GB avg-val = 96.23 avg-zval = 30.71 2024/11/23-04:16:15.995926 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244301.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:16:16.642768 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244301.sst): second pass time = 1.37's, 75.865'MB/sec, value only(77.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 57.070'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3332.516'MB/sec (index lex order gen) rebuild zvType time = 0.51's, 0.530'MB/sec write SST data time = 0.14's, 268.061'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.026'MB, throughput = 29986.027'MB/sec zip my value time = 1.37's, unzip length = 0.104'GB zip my value throughput = 75.865'MB/sec zip pipeline throughput = 75.865'MB/sec entries = 1080418 avg-key = 28.00 avg-zkey = 1.93 avg-val = 96.23 avg-zval = 30.71 usrkeys = 1080418 avg-key = 20.00 avg-zkey = 1.93 avg-val = 97.14 avg-zval = 30.71 TagRS{ kinds = RS_Key0_TagN bytes = 168920 } TagArray{ size = 141251 bytes = 494400 } seq expand size = 988757 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0216 GB value = 0.1040 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0332 GB dict = 1.03 MB all = 0.0370 GB } UnZip/Zip{ index = 10.3794 value = 3.1330 dict = 1.00 all = 3.6309 } Zip/UnZip{ index = 0.0963 value = 0.3192 dict = 1.00 all = 0.2754 } ---------------------------- total value len = 0.560409 GB avg = 0.117 KB (by entry num) total key len = 0.134615 GB avg = 0.028 KB total ukey len = 0.096154 GB avg = 0.020 KB total ukey num = 0.004807686 Billion total entry num = 0.004807686 Billion write speed all = 0.009952579 MB/sec (with seq num) write speed all = 0.009401820 MB/sec (without seq num) 2024/11/23-04:16:16.675809 140025171461696 [compaction_job.cc:1948] [default] [JOB 19094] Generated table #244301: 1080418 keys, 36965805 bytes, temperature: kWarm 2024/11/23-04:16:16.675875 140025171461696 EVENT_LOG_v1 {"time_micros": 1732306576675841, "cf_name": "default", "job": 19094, "event": "table_file_creation", "file_number": 244301, "file_size": 36965805, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3408555208, "table_properties": {"data_size": 33183776, "index_size": 2081856, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30251704, "raw_average_key_size": 28, "raw_value_size": 103966029, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1080418, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732306573, "oldest_key_time": 0, "file_creation_time": 1732306573, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 244301, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:16:16.713479 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244302.sst) /dev/shm/ToplingTemp/Topling-405930-rXN1CI 2024/11/23-04:16:17.607901 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244302.sst): first pass time = 0.89's, 150.067'MB/sec 2024/11/23-04:16:18.874627 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244302.sst): old prealloc_size = 147639500, real_size = 31006896 2024/11/23-04:16:18.875218 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244302.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924367 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0290 GB avg-val = 117.20 avg-zval = 31.38 2024/11/23-04:16:18.876017 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244302.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:16:19.733218 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244302.sst): second pass time = 1.27's, 85.575'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 41.013'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3125.369'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.316'MB/sec write SST data time = 0.13's, 278.894'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 32219.674'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 85.575'MB/sec zip pipeline throughput = 85.575'MB/sec entries = 924367 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.20 avg-zval = 31.38 usrkeys = 924367 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.20 avg-zval = 31.38 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924367 bytes = 3350848 } seq expand size = 6470569 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0290 GB dict = 1.08 MB all = 0.0352 GB } UnZip/Zip{ index = 10.4449 value = 3.7350 dict = 1.00 all = 3.8116 } Zip/UnZip{ index = 0.0957 value = 0.2677 dict = 1.00 all = 0.2624 } ---------------------------- total value len = 0.675215 GB avg = 0.118 KB (by entry num) total key len = 0.160497 GB avg = 0.028 KB total ukey len = 0.114641 GB avg = 0.020 KB total ukey num = 0.005732053 Billion total entry num = 0.005732053 Billion write speed all = 0.011966674 MB/sec (with seq num) write speed all = 0.011310050 MB/sec (without seq num) 2024/11/23-04:16:19.764387 140025171461696 [compaction_job.cc:1948] [default] [JOB 19094] Generated table #244302: 924367 keys, 35212544 bytes, temperature: kWarm 2024/11/23-04:16:19.764453 140025171461696 EVENT_LOG_v1 {"time_micros": 1732306579764418, "cf_name": "default", "job": 19094, "event": "table_file_creation", "file_number": 244302, "file_size": 35212544, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3408555209, "largest_seqno": 3780162686, "table_properties": {"data_size": 29005664, "index_size": 1769984, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25882276, "raw_average_key_size": 28, "raw_value_size": 108335534, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924367, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732306576, "oldest_key_time": 0, "file_creation_time": 1732306576, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 244302, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:16:19.807905 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244303.sst) /dev/shm/ToplingTemp/Topling-405930-VxPWFJ 2024/11/23-04:16:20.603535 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244303.sst): first pass time = 0.80's, 168.701'MB/sec 2024/11/23-04:16:21.940074 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244303.sst): old prealloc_size = 147639500, real_size = 32442368 2024/11/23-04:16:21.940607 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244303.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922480 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0305 GB avg-val = 117.50 avg-zval = 33.02 2024/11/23-04:16:21.941403 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244303.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:16:22.829677 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244303.sst): second pass time = 1.34's, 81.167'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 41.193'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3459.889'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.302'MB/sec write SST data time = 0.13's, 289.312'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 23805.965'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.167'MB/sec zip pipeline throughput = 81.167'MB/sec entries = 922480 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.50 avg-zval = 33.02 usrkeys = 922480 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.50 avg-zval = 33.02 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922480 bytes = 3344000 } seq expand size = 6457360 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5110 value = 3.5588 dict = 1.00 all = 3.6627 } Zip/UnZip{ index = 0.0951 value = 0.2810 dict = 1.00 all = 0.2730 } ---------------------------- total value len = 0.790061 GB avg = 0.119 KB (by entry num) total key len = 0.186327 GB avg = 0.028 KB total ukey len = 0.133091 GB avg = 0.020 KB total ukey num = 0.006654533 Billion total entry num = 0.006654533 Billion write speed all = 0.013980401 MB/sec (with seq num) write speed all = 0.013218138 MB/sec (without seq num) 2024/11/23-04:16:22.860371 140025171461696 [compaction_job.cc:1948] [default] [JOB 19094] Generated table #244303: 922480 keys, 36644716 bytes, temperature: kWarm 2024/11/23-04:16:22.860438 140025171461696 EVENT_LOG_v1 {"time_micros": 1732306582860403, "cf_name": "default", "job": 19094, "event": "table_file_creation", "file_number": 244303, "file_size": 36644716, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3487497056, "largest_seqno": 3780158819, "table_properties": {"data_size": 30456304, "index_size": 1755264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25829440, "raw_average_key_size": 28, "raw_value_size": 108388388, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922480, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732306579, "oldest_key_time": 0, "file_creation_time": 1732306579, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 244303, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:16:22.895089 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244304.sst) /dev/shm/ToplingTemp/Topling-405930-w9Lw0H 2024/11/23-04:16:23.787028 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244304.sst): first pass time = 0.89's, 150.486'MB/sec 2024/11/23-04:16:25.112387 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244304.sst): old prealloc_size = 147639500, real_size = 32674128 2024/11/23-04:16:25.112935 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244304.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931264 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0307 GB avg-val = 116.12 avg-zval = 32.93 2024/11/23-04:16:25.115212 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244304.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:16:26.068990 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244304.sst): second pass time = 1.32's, 81.665'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 38.576'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3369.709'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.281'MB/sec write SST data time = 0.13's, 288.904'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31789.256'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.665'MB/sec zip pipeline throughput = 81.665'MB/sec entries = 931264 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.12 avg-zval = 32.93 usrkeys = 931264 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.12 avg-zval = 32.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931264 bytes = 3375840 } seq expand size = 6518848 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5122 value = 3.5261 dict = 1.00 all = 3.6372 } Zip/UnZip{ index = 0.0951 value = 0.2836 dict = 1.00 all = 0.2749 } ---------------------------- total value len = 0.904722 GB avg = 0.119 KB (by entry num) total key len = 0.212402 GB avg = 0.028 KB total ukey len = 0.151716 GB avg = 0.020 KB total ukey num = 0.007585797 Billion total entry num = 0.007585797 Billion write speed all = 0.015994797 MB/sec (with seq num) write speed all = 0.015125899 MB/sec (without seq num) 2024/11/23-04:16:26.102156 140025171461696 [compaction_job.cc:1948] [default] [JOB 19094] Generated table #244304: 931264 keys, 36901220 bytes, temperature: kWarm 2024/11/23-04:16:26.102230 140025171461696 EVENT_LOG_v1 {"time_micros": 1732306586102186, "cf_name": "default", "job": 19094, "event": "table_file_creation", "file_number": 244304, "file_size": 36901220, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3375334738, "largest_seqno": 3780162980, "table_properties": {"data_size": 30669376, "index_size": 1771776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26075392, "raw_average_key_size": 28, "raw_value_size": 108142440, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931264, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732306582, "oldest_key_time": 0, "file_creation_time": 1732306582, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 244304, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:16:26.140520 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244305.sst) /dev/shm/ToplingTemp/Topling-405930-aOHxyJ 2024/11/23-04:16:27.022060 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244305.sst): first pass time = 0.88's, 152.260'MB/sec 2024/11/23-04:16:28.334915 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244305.sst): old prealloc_size = 147639500, real_size = 32916768 2024/11/23-04:16:28.335465 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244305.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923911 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 117.27 avg-zval = 33.46 2024/11/23-04:16:28.337152 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244305.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:16:29.255174 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244305.sst): second pass time = 1.31's, 82.595'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.341'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3358.370'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.292'MB/sec write SST data time = 0.13's, 288.297'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 31504.135'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.595'MB/sec zip pipeline throughput = 82.595'MB/sec entries = 923911 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.27 avg-zval = 33.46 usrkeys = 923911 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.27 avg-zval = 33.46 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923911 bytes = 3349200 } seq expand size = 6467377 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.4435 value = 3.5046 dict = 1.00 all = 3.6155 } Zip/UnZip{ index = 0.0958 value = 0.2853 dict = 1.00 all = 0.2766 } ---------------------------- total value len = 1.019537 GB avg = 0.120 KB (by entry num) total key len = 0.238272 GB avg = 0.028 KB total ukey len = 0.170194 GB avg = 0.020 KB total ukey num = 0.008509708 Billion total entry num = 0.008509708 Billion write speed all = 0.018008282 MB/sec (with seq num) write speed all = 0.017033601 MB/sec (without seq num) 2024/11/23-04:16:29.286607 140025171461696 [compaction_job.cc:1948] [default] [JOB 19094] Generated table #244305: 923911 keys, 37122826 bytes, temperature: kWarm 2024/11/23-04:16:29.286672 140025171461696 EVENT_LOG_v1 {"time_micros": 1732306589286639, "cf_name": "default", "job": 19094, "event": "table_file_creation", "file_number": 244305, "file_size": 37122826, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3780163083, "table_properties": {"data_size": 30916240, "index_size": 1769344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25869508, "raw_average_key_size": 28, "raw_value_size": 108348250, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923911, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732306586, "oldest_key_time": 0, "file_creation_time": 1732306586, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 244305, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:16:29.332147 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244306.sst) /dev/shm/ToplingTemp/Topling-405930-m9XBpJ 2024/11/23-04:16:30.217245 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244306.sst): first pass time = 0.89's, 151.650'MB/sec 2024/11/23-04:16:31.509520 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244306.sst): old prealloc_size = 147639500, real_size = 32797888 2024/11/23-04:16:31.510070 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244306.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930737 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0308 GB avg-val = 116.21 avg-zval = 33.07 2024/11/23-04:16:31.510871 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244306.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:16:32.465723 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244306.sst): second pass time = 1.29's, 83.738'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 38.476'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3377.709'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.281'MB/sec write SST data time = 0.13's, 289.404'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 30957.131'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.738'MB/sec zip pipeline throughput = 83.738'MB/sec entries = 930737 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.21 avg-zval = 33.07 usrkeys = 930737 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.21 avg-zval = 33.07 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930737 bytes = 3141248 } seq expand size = 6515159 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.4440 value = 3.5136 dict = 1.00 all = 3.6482 } Zip/UnZip{ index = 0.0957 value = 0.2846 dict = 1.00 all = 0.2741 } ---------------------------- total value len = 1.134210 GB avg = 0.120 KB (by entry num) total key len = 0.264332 GB avg = 0.028 KB total ukey len = 0.188809 GB avg = 0.020 KB total ukey num = 0.009440445 Billion total entry num = 0.009440445 Billion write speed all = 0.020022260 MB/sec (with seq num) write speed all = 0.018941025 MB/sec (without seq num) 2024/11/23-04:16:32.498538 140025171461696 [compaction_job.cc:1948] [default] [JOB 19094] Generated table #244306: 930737 keys, 36790416 bytes, temperature: kWarm 2024/11/23-04:16:32.498605 140025171461696 EVENT_LOG_v1 {"time_micros": 1732306592498570, "cf_name": "default", "job": 19094, "event": "table_file_creation", "file_number": 244306, "file_size": 36790416, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3680980241, "largest_seqno": 3780162869, "table_properties": {"data_size": 30782704, "index_size": 1782336, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26060636, "raw_average_key_size": 28, "raw_value_size": 108157138, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930737, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732306589, "oldest_key_time": 0, "file_creation_time": 1732306589, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 244306, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:16:32.546144 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244307.sst) /dev/shm/ToplingTemp/Topling-405930-tMBTxK 2024/11/23-04:16:33.487741 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244307.sst): first pass time = 0.94's, 142.548'MB/sec 2024/11/23-04:16:34.813800 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244307.sst): old prealloc_size = 147639500, real_size = 32870896 2024/11/23-04:16:34.814346 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244307.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924910 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 117.11 avg-zval = 33.39 2024/11/23-04:16:34.816080 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244307.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:16:35.737615 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244307.sst): second pass time = 1.33's, 81.729'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.137'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3358.737'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.290'MB/sec write SST data time = 0.13's, 292.728'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 29203.718'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.729'MB/sec zip pipeline throughput = 81.729'MB/sec entries = 924910 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.11 avg-zval = 33.39 usrkeys = 924910 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.11 avg-zval = 33.39 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924910 bytes = 3352816 } seq expand size = 6474370 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5142 value = 3.5078 dict = 1.00 all = 3.6198 } Zip/UnZip{ index = 0.0951 value = 0.2851 dict = 1.00 all = 0.2763 } ---------------------------- total value len = 1.249004 GB avg = 0.120 KB (by entry num) total key len = 0.290230 GB avg = 0.028 KB total ukey len = 0.207307 GB avg = 0.020 KB total ukey num = 0.010365355 Billion total entry num = 0.010365355 Billion write speed all = 0.022035450 MB/sec (with seq num) write speed all = 0.020848339 MB/sec (without seq num) 2024/11/23-04:16:35.768128 140025171461696 [compaction_job.cc:1948] [default] [JOB 19094] Generated table #244307: 924910 keys, 37079030 bytes, temperature: kWarm 2024/11/23-04:16:35.768192 140025171461696 EVENT_LOG_v1 {"time_micros": 1732306595768160, "cf_name": "default", "job": 19094, "event": "table_file_creation", "file_number": 244307, "file_size": 37079030, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369194706, "largest_seqno": 3780161287, "table_properties": {"data_size": 30880096, "index_size": 1759360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25897480, "raw_average_key_size": 28, "raw_value_size": 108320280, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924910, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732306592, "oldest_key_time": 0, "file_creation_time": 1732306592, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 244307, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:16:35.807172 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244308.sst) /dev/shm/ToplingTemp/Topling-405930-rVoXWK 2024/11/23-04:16:36.607146 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244308.sst): first pass time = 0.80's, 167.785'MB/sec 2024/11/23-04:16:37.912236 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244308.sst): old prealloc_size = 147639500, real_size = 32660416 2024/11/23-04:16:37.912773 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244308.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931799 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0306 GB avg-val = 116.04 avg-zval = 32.89 2024/11/23-04:16:37.914479 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244308.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:16:38.873133 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244308.sst): second pass time = 1.30's, 82.906'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 38.391'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3382.851'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.280'MB/sec write SST data time = 0.13's, 287.615'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31267.638'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 82.906'MB/sec zip pipeline throughput = 82.906'MB/sec entries = 931799 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.04 avg-zval = 32.89 usrkeys = 931799 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.04 avg-zval = 32.89 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931799 bytes = 3377792 } seq expand size = 6522593 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4424 value = 3.5287 dict = 1.00 all = 3.6384 } Zip/UnZip{ index = 0.0958 value = 0.2834 dict = 1.00 all = 0.2748 } ---------------------------- total value len = 1.363654 GB avg = 0.121 KB (by entry num) total key len = 0.316320 GB avg = 0.028 KB total ukey len = 0.225943 GB avg = 0.020 KB total ukey num = 0.011297154 Billion total entry num = 0.011297154 Billion write speed all = 0.024049189 MB/sec (with seq num) write speed all = 0.022755420 MB/sec (without seq num) 2024/11/23-04:16:38.904601 140025171461696 [compaction_job.cc:1948] [default] [JOB 19094] Generated table #244308: 931799 keys, 36889074 bytes, temperature: kWarm 2024/11/23-04:16:38.904668 140025171461696 EVENT_LOG_v1 {"time_micros": 1732306598904634, "cf_name": "default", "job": 19094, "event": "table_file_creation", "file_number": 244308, "file_size": 36889074, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3339428666, "largest_seqno": 3780163033, "table_properties": {"data_size": 30642672, "index_size": 1784640, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26090372, "raw_average_key_size": 28, "raw_value_size": 108127478, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931799, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732306595, "oldest_key_time": 0, "file_creation_time": 1732306595, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 244308, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:16:38.967799 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244309.sst) /dev/shm/ToplingTemp/Topling-405930-WtJTSH 2024/11/23-04:16:39.821794 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244309.sst): first pass time = 0.85's, 157.173'MB/sec 2024/11/23-04:16:41.165384 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244309.sst): old prealloc_size = 147639500, real_size = 33066304 2024/11/23-04:16:41.165929 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244309.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931762 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0311 GB avg-val = 116.05 avg-zval = 33.34 2024/11/23-04:16:41.167564 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244309.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:16:42.083850 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244309.sst): second pass time = 1.34's, 80.525'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.479'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3399.898'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.294'MB/sec write SST data time = 0.13's, 294.206'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 30324.300'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.525'MB/sec zip pipeline throughput = 80.525'MB/sec entries = 931762 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.05 avg-zval = 33.34 usrkeys = 931762 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.05 avg-zval = 33.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931762 bytes = 3261184 } seq expand size = 6522334 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5129 value = 3.4812 dict = 1.00 all = 3.6101 } Zip/UnZip{ index = 0.0951 value = 0.2873 dict = 1.00 all = 0.2770 } ---------------------------- total value len = 1.478305 GB avg = 0.121 KB (by entry num) total key len = 0.342410 GB avg = 0.028 KB total ukey len = 0.244578 GB avg = 0.020 KB total ukey num = 0.012228916 Billion total entry num = 0.012228916 Billion write speed all = 0.026062715 MB/sec (with seq num) write speed all = 0.024662303 MB/sec (without seq num) 2024/11/23-04:16:42.115945 140025171461696 [compaction_job.cc:1948] [default] [JOB 19094] Generated table #244309: 931762 keys, 37178006 bytes, temperature: kWarm 2024/11/23-04:16:42.116021 140025171461696 EVENT_LOG_v1 {"time_micros": 1732306602115977, "cf_name": "default", "job": 19094, "event": "table_file_creation", "file_number": 244309, "file_size": 37178006, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 3780162788, "table_properties": {"data_size": 31060592, "index_size": 1772608, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26089336, "raw_average_key_size": 28, "raw_value_size": 108128404, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931762, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732306598, "oldest_key_time": 0, "file_creation_time": 1732306598, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 244309, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:16:42.163959 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244310.sst) /dev/shm/ToplingTemp/Topling-405930-3Stf9K 2024/11/23-04:16:42.970904 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244310.sst): first pass time = 0.81's, 166.338'MB/sec 2024/11/23-04:16:44.296582 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244310.sst): old prealloc_size = 147639500, real_size = 33168208 2024/11/23-04:16:44.297125 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244310.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927651 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.69 avg-zval = 33.59 2024/11/23-04:16:44.300280 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244310.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:16:45.238280 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244310.sst): second pass time = 1.32's, 81.696'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.701'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3409.464'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.286'MB/sec write SST data time = 0.13's, 288.169'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33722.732'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.696'MB/sec zip pipeline throughput = 81.696'MB/sec entries = 927651 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.69 avg-zval = 33.59 usrkeys = 927651 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.69 avg-zval = 33.59 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927651 bytes = 3362752 } seq expand size = 6493557 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4431 value = 3.4738 dict = 1.00 all = 3.5901 } Zip/UnZip{ index = 0.0958 value = 0.2879 dict = 1.00 all = 0.2785 } ---------------------------- total value len = 1.593042 GB avg = 0.121 KB (by entry num) total key len = 0.368384 GB avg = 0.028 KB total ukey len = 0.263131 GB avg = 0.020 KB total ukey num = 0.013156567 Billion total entry num = 0.013156567 Billion write speed all = 0.028075667 MB/sec (with seq num) write speed all = 0.026569092 MB/sec (without seq num) 2024/11/23-04:16:45.272957 140025171461696 [compaction_job.cc:1948] [default] [JOB 19094] Generated table #244310: 927651 keys, 37385250 bytes, temperature: kWarm 2024/11/23-04:16:45.273023 140025171461696 EVENT_LOG_v1 {"time_micros": 1732306605272989, "cf_name": "default", "job": 19094, "event": "table_file_creation", "file_number": 244310, "file_size": 37385250, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 3780162798, "table_properties": {"data_size": 31159552, "index_size": 1776576, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25974228, "raw_average_key_size": 28, "raw_value_size": 108243542, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927651, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732306602, "oldest_key_time": 0, "file_creation_time": 1732306602, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 244310, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:16:45.306655 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244311.sst) /dev/shm/ToplingTemp/Topling-405930-TVysFG 2024/11/23-04:16:45.665091 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244311.sst): first pass time = 0.36's, 252.952'MB/sec 2024/11/23-04:16:46.551547 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244311.sst): old prealloc_size = 147639500, real_size = 22847872 2024/11/23-04:16:46.551985 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244311.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 631110 prefix = 4 raw-key = 0.0101 GB zip-key = 0.0012 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0730 GB zip-val = 0.0215 GB avg-val = 115.65 avg-zval = 34.04 2024/11/23-04:16:46.552415 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244311.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0016 GB, waited 0.000 sec, Key+Value = 0.091 GB 2024/11/23-04:16:47.154913 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00/244311.sst): second pass time = 0.89's, 82.396'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.60's, 41.543'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2903.493'MB/sec (index lex order gen) rebuild zvType time = 0.51's, 0.307'MB/sec write SST data time = 0.09's, 279.693'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.733'MB, throughput = 24058.572'MB/sec zip my value time = 0.89's, unzip length = 0.073'GB zip my value throughput = 82.396'MB/sec zip pipeline throughput = 82.396'MB/sec entries = 631110 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.65 avg-zval = 34.04 usrkeys = 631110 avg-key = 20.00 avg-zkey = 1.92 avg-val = 120.89 avg-zval = 34.04 TagRS{ kinds = RS_Key0_TagN bytes = 98720 } TagArray{ size = 472732 bytes = 1536400 } seq expand size = 3309124 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0126 GB value = 0.0730 GB dict = 0.73 MB all = 0.0907 GB } __ZipSize{ index = 0.0012 GB value = 0.0215 GB dict = 0.73 MB all = 0.0251 GB } UnZip/Zip{ index = 10.4428 value = 3.3978 dict = 1.00 all = 3.6170 } Zip/UnZip{ index = 0.0958 value = 0.2943 dict = 1.00 all = 0.2765 } ---------------------------- total value len = 1.669339 GB avg = 0.121 KB (by entry num) total key len = 0.386055 GB avg = 0.028 KB total ukey len = 0.275754 GB avg = 0.020 KB total ukey num = 0.013787677 Billion total entry num = 0.013787677 Billion write speed all = 0.029419909 MB/sec (with seq num) write speed all = 0.027841108 MB/sec (without seq num) 2024/11/23-04:16:47.179744 140025171461696 [compaction_job.cc:1948] [default] [JOB 19094] Generated table #244311: 631110 keys, 25064887 bytes, temperature: kWarm 2024/11/23-04:16:47.179813 140025171461696 EVENT_LOG_v1 {"time_micros": 1732306607179774, "cf_name": "default", "job": 19094, "event": "table_file_creation", "file_number": 244311, "file_size": 25064887, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3780162810, "table_properties": {"data_size": 21481200, "index_size": 1208704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 17671080, "raw_average_key_size": 28, "raw_value_size": 72987824, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 631110, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732306605, "oldest_key_time": 0, "file_creation_time": 1732306605, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 244311, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:16:47.200245 140025171461696 [compaction_job.cc:703] job-19094: subcompact[0], size: 0.542514 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-19094/att-00 [244297,244298,244299,244300,244301,244302,244303,244304,244305,244306,244307,244308,244309,244310,244311] 2024/11/23-04:16:47.292550 140025171461696 (Original Log Time 2024/11/23-04:15:59.037776) EVENT_LOG_v1 {"time_micros": 1732306559037744, "job": 19094, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [244245], "files_L3": [244293, 244294, 244196, 244197, 244198, 244199, 244200, 244201, 244202, 244203, 244204, 244205, 244159, 244160, 244161, 244274], "score": 0.995684, "input_data_size": 563060414, "oldest_snapshot_seqno": 3780523968} 2024/11/23-04:16:47.292554 140025171461696 (Original Log Time 2024/11/23-04:16:47.291805) [compaction_job.cc:2022] [default] [JOB 19094] Compacted 1@2 + 16@3 files to L3 => 542513597 bytes 2024/11/23-04:16:47.292555 140025171461696 (Original Log Time 2024/11/23-04:16:47.292484) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 11.7 rd, 11.3 wr, level 3, files in(1, 16) out(15 +0 blob) MB in(27.1, 509.9 +0.0 blob) out(517.4 +0.0 blob), read-write-amplify(38.9) write-amplify(19.1) OK, records in: 13874721, records dropped: 87044 output_compression: Snappy 2024/11/23-04:16:47.292561 140025171461696 (Original Log Time 2024/11/23-04:16:47.292529) EVENT_LOG_v1 {"time_micros": 1732306607292507, "cf": "default", "job": 19094, "event": "compaction_finished", "compaction_time_micros": 48162470, "compaction_time_cpu_micros": 21523774, "output_level": 3, "num_output_files": 15, "total_output_size": 542513597, "num_input_records": 13874721, "num_output_records": 13787677, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 359305688, "file_range_sync_nanos": 0, "file_fsync_nanos": 14353, "file_prepare_write_nanos": 31672, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/23-04:16:47.418603 140025171461696 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732306558,"executesMs":48410,"compactionJobId":19094,"attempt":0,"compactionInputRawBytes":1982068211,"compactionInputZipBytes":563060414,"compactionOutputRawBytes":0,"compactionOutputZipBytes":542513597,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-a6a0ab09f658a813fae4bff746cb0610-a751ccef09a8c9b8-00"}}, response = 2024/11/23-04:16:47.418671 140025171461696 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-19094/att-00: olev 3, work 48.411 s, result 4.842 ms, install 0.783 ms, input{raw 1.846 GiB zip 536.976 MiB}