2024/11/23-06:21:18.222414 140025129498176 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00: fork to child time = 0.009519 sec 2024/11/23-06:21:18.223326 140025129498176 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00 2024/11/23-06:21:18.230133 140025129498176 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00 2024/11/23-06:21:18.231237 140025129498176 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-06:21:18.231523 140025129498176 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-06:21:18.296900 140025129498176 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-20937/att-00/MANIFEST-206347 2024/11/23-06:21:18.326822 140025129498176 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-20937/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 248338, last_sequence is 3832953367, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 248207 2024/11/23-06:21:18.326830 140025129498176 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-06:21:18.327176 140025129498176 [version_set.cc:5721] Creating manifest 248338 2024/11/23-06:21:18.378433 140025129498176 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-06:21:18.398873 140025129498176 [compaction_job.cc:2437] [default] [JOB 20937] Compacting 1@2 + 12@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-06:21:18.398898 140025129498176 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 15669 Base level 2, inputs: [248333(27MB)], [248244(36MB) 248245(36MB) 248247(34MB) 248248(34MB) 248249(34MB) 248268(33MB) 248269(33MB) 248270(36MB) 248271(35MB) 248272(33MB) 248273(35MB) 248274(35MB)] 2024/11/23-06:21:18.409136 140025129498176 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-06:21:18.409179 140025129498176 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-06:21:18.409208 140025129498176 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-06:21:18.409216 140025129498176 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-06:21:18.409231 140025129498176 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-06:21:18.897263 140025129498176 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248339.sst) /dev/shm/ToplingTemp/Topling-433003-wy0ehP 2024/11/23-06:21:19.481658 140025129498176 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248339.sst): first pass time = 0.58's, 229.685'MB/sec 2024/11/23-06:21:21.465966 140025129498176 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248339.sst): old prealloc_size = 147639500, real_size = 34164896 2024/11/23-06:21:21.466543 140025129498176 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248339.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929078 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0322 GB avg-val = 116.46 avg-zval = 34.62 2024/11/23-06:21:21.467381 140025129498176 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248339.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:21:23.066748 140025129498176 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248339.sst): second pass time = 1.98's, 54.550'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.60's, 23.980'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3165.914'MB/sec (index lex order gen) rebuild zvType time = 1.33's, 0.174'MB/sec write SST data time = 0.27's, 144.467'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31450.779'MB/sec zip my value time = 1.98's, unzip length = 0.108'GB zip my value throughput = 54.550'MB/sec zip pipeline throughput = 145.167'MB/sec entries = 929078 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.46 avg-zval = 34.62 usrkeys = 929078 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.46 avg-zval = 34.62 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929078 bytes = 3367920 } seq expand size = 6503546 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0322 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.5126 value = 3.3640 dict = 1.00 all = 3.4965 } Zip/UnZip{ index = 0.0951 value = 0.2973 dict = 1.00 all = 0.2860 } ---------------------------- total value len = 0.114707 GB avg = 0.123 KB (by entry num) total key len = 0.026014 GB avg = 0.028 KB total ukey len = 0.018582 GB avg = 0.020 KB total ukey num = 0.000929078 Billion total entry num = 0.000929078 Billion write speed all = 0.001819516 MB/sec (with seq num) write speed all = 0.001723413 MB/sec (without seq num) 2024/11/23-06:21:23.116158 140025129498176 [compaction_job.cc:1948] [default] [JOB 20937] Generated table #248339: 929078 keys, 38385868 bytes, temperature: kWarm 2024/11/23-06:21:23.116271 140025129498176 EVENT_LOG_v1 {"time_micros": 1732314083116201, "cf_name": "default", "job": 20937, "event": "table_file_creation", "file_number": 248339, "file_size": 38385868, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3433383143, "largest_seqno": 3832926900, "table_properties": {"data_size": 32164880, "index_size": 1767552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26014184, "raw_average_key_size": 28, "raw_value_size": 108203568, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929078, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732314078, "oldest_key_time": 0, "file_creation_time": 1732314078, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248339, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:21:23.187031 140025129498176 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248340.sst) /dev/shm/ToplingTemp/Topling-433003-7byUUQ 2024/11/23-06:21:24.084918 140025129498176 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248340.sst): first pass time = 0.90's, 149.491'MB/sec 2024/11/23-06:21:25.717698 140025129498176 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248340.sst): old prealloc_size = 147639500, real_size = 33557584 2024/11/23-06:21:25.718282 140025129498176 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248340.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933954 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0315 GB avg-val = 115.71 avg-zval = 33.78 2024/11/23-06:21:25.719140 140025129498176 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248340.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:21:27.276139 140025129498176 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248340.sst): second pass time = 1.63's, 66.214'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.56's, 24.326'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3199.441'MB/sec (index lex order gen) rebuild zvType time = 1.40's, 0.167'MB/sec write SST data time = 0.16's, 236.786'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 43024.984'MB/sec zip my value time = 1.63's, unzip length = 0.108'GB zip my value throughput = 66.214'MB/sec zip pipeline throughput = 132.528'MB/sec entries = 933954 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.71 avg-zval = 33.78 usrkeys = 933954 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.71 avg-zval = 33.78 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933954 bytes = 3502336 } seq expand size = 6537678 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.5111 value = 3.4256 dict = 1.00 all = 3.5404 } Zip/UnZip{ index = 0.0951 value = 0.2919 dict = 1.00 all = 0.2825 } ---------------------------- total value len = 0.229312 GB avg = 0.123 KB (by entry num) total key len = 0.052165 GB avg = 0.028 KB total ukey len = 0.037261 GB avg = 0.020 KB total ukey num = 0.001863032 Billion total entry num = 0.001863032 Billion write speed all = 0.003639276 MB/sec (with seq num) write speed all = 0.003446575 MB/sec (without seq num) 2024/11/23-06:21:27.312216 140025129498176 [compaction_job.cc:1948] [default] [JOB 20937] Generated table #248340: 933954 keys, 37910370 bytes, temperature: kWarm 2024/11/23-06:21:27.312279 140025129498176 EVENT_LOG_v1 {"time_micros": 1732314087312248, "cf_name": "default", "job": 20937, "event": "table_file_creation", "file_number": 248340, "file_size": 37910370, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3832927066, "table_properties": {"data_size": 31546816, "index_size": 1777088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26150712, "raw_average_key_size": 28, "raw_value_size": 108067108, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933954, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732314083, "oldest_key_time": 0, "file_creation_time": 1732314083, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248340, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:21:27.350822 140025129498176 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248341.sst) /dev/shm/ToplingTemp/Topling-433003-VNOjER 2024/11/23-06:21:28.223717 140025129498176 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248341.sst): first pass time = 0.87's, 153.768'MB/sec 2024/11/23-06:21:29.679739 140025129498176 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248341.sst): old prealloc_size = 147639500, real_size = 31843712 2024/11/23-06:21:29.680314 140025129498176 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248341.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927196 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0298 GB avg-val = 116.76 avg-zval = 32.19 2024/11/23-06:21:29.681156 140025129498176 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248341.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:21:31.170792 140025129498176 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248341.sst): second pass time = 1.46's, 74.388'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.49's, 24.184'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3199.884'MB/sec (index lex order gen) rebuild zvType time = 1.29's, 0.180'MB/sec write SST data time = 0.20's, 180.784'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 26355.920'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 74.388'MB/sec zip pipeline throughput = 74.388'MB/sec entries = 927196 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.76 avg-zval = 32.19 usrkeys = 927196 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.76 avg-zval = 32.19 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927196 bytes = 3361104 } seq expand size = 6490372 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0298 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.5122 value = 3.6270 dict = 1.00 all = 3.7221 } Zip/UnZip{ index = 0.0951 value = 0.2757 dict = 1.00 all = 0.2687 } ---------------------------- total value len = 0.344059 GB avg = 0.123 KB (by entry num) total key len = 0.078126 GB avg = 0.028 KB total ukey len = 0.055805 GB avg = 0.020 KB total ukey num = 0.002790228 Billion total entry num = 0.002790228 Billion write speed all = 0.005458247 MB/sec (with seq num) write speed all = 0.005169658 MB/sec (without seq num) 2024/11/23-06:21:31.203832 140025129498176 [compaction_job.cc:1948] [default] [JOB 20937] Generated table #248341: 927196 keys, 36059436 bytes, temperature: kWarm 2024/11/23-06:21:31.203901 140025129498176 EVENT_LOG_v1 {"time_micros": 1732314091203861, "cf_name": "default", "job": 20937, "event": "table_file_creation", "file_number": 248341, "file_size": 36059436, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369194706, "largest_seqno": 3832926953, "table_properties": {"data_size": 29847728, "index_size": 1764032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25961488, "raw_average_key_size": 28, "raw_value_size": 108256240, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927196, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732314087, "oldest_key_time": 0, "file_creation_time": 1732314087, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248341, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:21:31.239004 140025129498176 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248342.sst) /dev/shm/ToplingTemp/Topling-433003-kxIa3Q 2024/11/23-06:21:32.148285 140025129498176 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248342.sst): first pass time = 0.91's, 147.615'MB/sec 2024/11/23-06:21:33.566844 140025129498176 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248342.sst): old prealloc_size = 147639500, real_size = 31437184 2024/11/23-06:21:33.567389 140025129498176 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248342.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928441 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0294 GB avg-val = 116.56 avg-zval = 31.71 2024/11/23-06:21:33.568202 140025129498176 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248342.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:21:34.450532 140025129498176 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248342.sst): second pass time = 1.42's, 76.333'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 40.350'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3360.488'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.308'MB/sec write SST data time = 0.13's, 274.628'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35927.911'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 76.333'MB/sec zip pipeline throughput = 76.333'MB/sec entries = 928441 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.56 avg-zval = 31.71 usrkeys = 928441 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.56 avg-zval = 31.71 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928441 bytes = 3365616 } seq expand size = 6499087 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5122 value = 3.6762 dict = 1.00 all = 3.7642 } Zip/UnZip{ index = 0.0951 value = 0.2720 dict = 1.00 all = 0.2657 } ---------------------------- total value len = 0.458779 GB avg = 0.123 KB (by entry num) total key len = 0.104123 GB avg = 0.028 KB total ukey len = 0.074373 GB avg = 0.020 KB total ukey num = 0.003718669 Billion total entry num = 0.003718669 Billion write speed all = 0.007277206 MB/sec (with seq num) write speed all = 0.006892606 MB/sec (without seq num) 2024/11/23-06:21:34.483767 140025129498176 [compaction_job.cc:1948] [default] [JOB 20937] Generated table #248342: 928441 keys, 35656018 bytes, temperature: kWarm 2024/11/23-06:21:34.483836 140025129498176 EVENT_LOG_v1 {"time_micros": 1732314094483805, "cf_name": "default", "job": 20937, "event": "table_file_creation", "file_number": 248342, "file_size": 35656018, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 3832927158, "table_properties": {"data_size": 29438512, "index_size": 1766400, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25996348, "raw_average_key_size": 28, "raw_value_size": 108221406, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928441, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732314091, "oldest_key_time": 0, "file_creation_time": 1732314091, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248342, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:21:34.517856 140025129498176 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248343.sst) /dev/shm/ToplingTemp/Topling-433003-LrgNqP 2024/11/23-06:21:35.402507 140025129498176 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248343.sst): first pass time = 0.88's, 151.725'MB/sec 2024/11/23-06:21:36.810263 140025129498176 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248343.sst): old prealloc_size = 147639500, real_size = 31035488 2024/11/23-06:21:36.810776 140025129498176 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248343.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928453 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0290 GB avg-val = 116.56 avg-zval = 31.27 2024/11/23-06:21:36.811611 140025129498176 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248343.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:21:37.672020 140025129498176 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248343.sst): second pass time = 1.41's, 76.911'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 40.910'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3497.916'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 273.225'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 36456.535'MB/sec zip my value time = 1.41's, unzip length = 0.108'GB zip my value throughput = 76.911'MB/sec zip pipeline throughput = 76.911'MB/sec entries = 928453 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.56 avg-zval = 31.27 usrkeys = 928453 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.56 avg-zval = 31.27 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928453 bytes = 3365664 } seq expand size = 6499171 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0290 GB dict = 1.08 MB all = 0.0353 GB } UnZip/Zip{ index = 10.5116 value = 3.7270 dict = 1.00 all = 3.8071 } Zip/UnZip{ index = 0.0951 value = 0.2683 dict = 1.00 all = 0.2627 } ---------------------------- total value len = 0.573499 GB avg = 0.123 KB (by entry num) total key len = 0.130119 GB avg = 0.028 KB total ukey len = 0.092942 GB avg = 0.020 KB total ukey num = 0.004647122 Billion total entry num = 0.004647122 Billion write speed all = 0.009096019 MB/sec (with seq num) write speed all = 0.008615414 MB/sec (without seq num) 2024/11/23-06:21:37.704213 140025129498176 [compaction_job.cc:1948] [default] [JOB 20937] Generated table #248343: 928453 keys, 35254342 bytes, temperature: kWarm 2024/11/23-06:21:37.704283 140025129498176 EVENT_LOG_v1 {"time_micros": 1732314097704250, "cf_name": "default", "job": 20937, "event": "table_file_creation", "file_number": 248343, "file_size": 35254342, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 3832926979, "table_properties": {"data_size": 29036688, "index_size": 1766528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25996684, "raw_average_key_size": 28, "raw_value_size": 108221090, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928453, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732314094, "oldest_key_time": 0, "file_creation_time": 1732314094, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248343, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:21:37.778914 140025129498176 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248344.sst) /dev/shm/ToplingTemp/Topling-433003-qaBUiQ 2024/11/23-06:21:38.644649 140025129498176 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248344.sst): first pass time = 0.87's, 155.040'MB/sec 2024/11/23-06:21:40.058793 140025129498176 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248344.sst): old prealloc_size = 147639500, real_size = 30802352 2024/11/23-06:21:40.059355 140025129498176 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248344.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926881 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0288 GB avg-val = 116.81 avg-zval = 31.07 2024/11/23-06:21:40.060174 140025129498176 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248344.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:21:40.923666 140025129498176 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248344.sst): second pass time = 1.41's, 76.620'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 40.486'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3289.418'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.13's, 272.522'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 37424.454'MB/sec zip my value time = 1.41's, unzip length = 0.108'GB zip my value throughput = 76.620'MB/sec zip pipeline throughput = 76.620'MB/sec entries = 926881 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.81 avg-zval = 31.07 usrkeys = 926881 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.81 avg-zval = 31.07 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926881 bytes = 3359952 } seq expand size = 6488167 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0288 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.4446 value = 3.7598 dict = 1.00 all = 3.8331 } Zip/UnZip{ index = 0.0957 value = 0.2660 dict = 1.00 all = 0.2609 } ---------------------------- total value len = 0.688253 GB avg = 0.123 KB (by entry num) total key len = 0.156072 GB avg = 0.028 KB total ukey len = 0.111480 GB avg = 0.020 KB total ukey num = 0.005574003 Billion total entry num = 0.005574003 Billion write speed all = 0.010914535 MB/sec (with seq num) write speed all = 0.010338096 MB/sec (without seq num) 2024/11/23-06:21:40.959554 140025129498176 [compaction_job.cc:1948] [default] [JOB 20937] Generated table #248344: 926881 keys, 35015052 bytes, temperature: kWarm 2024/11/23-06:21:40.959619 140025129498176 EVENT_LOG_v1 {"time_micros": 1732314100959585, "cf_name": "default", "job": 20937, "event": "table_file_creation", "file_number": 248344, "file_size": 35015052, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3524265952, "largest_seqno": 3832926998, "table_properties": {"data_size": 28795616, "index_size": 1774848, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25952668, "raw_average_key_size": 28, "raw_value_size": 108265174, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926881, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732314097, "oldest_key_time": 0, "file_creation_time": 1732314097, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248344, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:21:41.001079 140025129498176 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248345.sst) /dev/shm/ToplingTemp/Topling-433003-qYTg4S 2024/11/23-06:21:41.847279 140025129498176 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248345.sst): first pass time = 0.85's, 158.619'MB/sec 2024/11/23-06:21:43.222956 140025129498176 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248345.sst): old prealloc_size = 147639500, real_size = 31813248 2024/11/23-06:21:43.223499 140025129498176 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248345.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923491 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0298 GB avg-val = 117.34 avg-zval = 32.28 2024/11/23-06:21:43.224327 140025129498176 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248345.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:21:44.092368 140025129498176 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248345.sst): second pass time = 1.38's, 78.807'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 41.429'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3410.443'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.312'MB/sec write SST data time = 0.13's, 281.530'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33994.599'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.807'MB/sec zip pipeline throughput = 81.969'MB/sec entries = 923491 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.34 avg-zval = 32.28 usrkeys = 923491 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.34 avg-zval = 32.28 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923491 bytes = 3347664 } seq expand size = 6464437 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0298 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.4456 value = 3.6345 dict = 1.00 all = 3.7264 } Zip/UnZip{ index = 0.0957 value = 0.2751 dict = 1.00 all = 0.2684 } ---------------------------- total value len = 0.803077 GB avg = 0.124 KB (by entry num) total key len = 0.181930 GB avg = 0.028 KB total ukey len = 0.129950 GB avg = 0.020 KB total ukey num = 0.006497494 Billion total entry num = 0.006497494 Billion write speed all = 0.012732605 MB/sec (with seq num) write speed all = 0.012060691 MB/sec (without seq num) 2024/11/23-06:21:44.124647 140025129498176 [compaction_job.cc:1948] [default] [JOB 20937] Generated table #248345: 923491 keys, 36018514 bytes, temperature: kWarm 2024/11/23-06:21:44.124709 140025129498176 EVENT_LOG_v1 {"time_micros": 1732314104124678, "cf_name": "default", "job": 20937, "event": "table_file_creation", "file_number": 248345, "file_size": 36018514, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 3832927017, "table_properties": {"data_size": 29814000, "index_size": 1768192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25857748, "raw_average_key_size": 28, "raw_value_size": 108360114, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923491, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732314100, "oldest_key_time": 0, "file_creation_time": 1732314100, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248345, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:21:44.158050 140025129498176 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248346.sst) /dev/shm/ToplingTemp/Topling-433003-On0HmR 2024/11/23-06:21:45.016622 140025129498176 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248346.sst): first pass time = 0.86's, 156.333'MB/sec 2024/11/23-06:21:46.550388 140025129498176 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248346.sst): old prealloc_size = 147639500, real_size = 34807040 2024/11/23-06:21:46.550932 140025129498176 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248346.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927794 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0328 GB avg-val = 116.66 avg-zval = 35.34 2024/11/23-06:21:46.551655 140025129498176 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248346.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:21:47.436889 140025129498176 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248346.sst): second pass time = 1.53's, 70.602'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 43.545'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3428.142'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.307'MB/sec write SST data time = 0.13's, 295.733'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 29783.981'MB/sec zip my value time = 1.53's, unzip length = 0.108'GB zip my value throughput = 70.602'MB/sec zip pipeline throughput = 95.472'MB/sec entries = 927794 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.66 avg-zval = 35.34 usrkeys = 927794 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.28 avg-zval = 35.34 TagRS{ kinds = RS_Key0_TagN bytes = 145080 } TagArray{ size = 744797 bytes = 2793008 } seq expand size = 5213579 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0328 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.3771 value = 3.3013 dict = 1.00 all = 3.4769 } Zip/UnZip{ index = 0.0964 value = 0.3029 dict = 1.00 all = 0.2876 } ---------------------------- total value len = 0.916530 GB avg = 0.123 KB (by entry num) total key len = 0.207908 GB avg = 0.028 KB total ukey len = 0.148506 GB avg = 0.020 KB total ukey num = 0.007425288 Billion total entry num = 0.007425288 Billion write speed all = 0.014534324 MB/sec (with seq num) write speed all = 0.013766498 MB/sec (without seq num) 2024/11/23-06:21:47.472859 140025129498176 [compaction_job.cc:1948] [default] [JOB 20937] Generated table #248346: 927794 keys, 38602552 bytes, temperature: kWarm 2024/11/23-06:21:47.472917 140025129498176 EVENT_LOG_v1 {"time_micros": 1732314107472887, "cf_name": "default", "job": 20937, "event": "table_file_creation", "file_number": 248346, "file_size": 38602552, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3832922607, "table_properties": {"data_size": 32786736, "index_size": 1788160, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25978232, "raw_average_key_size": 28, "raw_value_size": 108239556, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927794, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732314104, "oldest_key_time": 0, "file_creation_time": 1732314104, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248346, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:21:47.506340 140025129498176 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248347.sst) /dev/shm/ToplingTemp/Topling-433003-c13BVR 2024/11/23-06:21:48.317272 140025129498176 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248347.sst): first pass time = 0.81's, 165.518'MB/sec 2024/11/23-06:21:49.908007 140025129498176 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248347.sst): old prealloc_size = 147639500, real_size = 32178560 2024/11/23-06:21:49.908551 140025129498176 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248347.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 936569 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0302 GB avg-val = 115.31 avg-zval = 32.21 2024/11/23-06:21:49.909351 140025129498176 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248347.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:21:50.855578 140025129498176 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248347.sst): second pass time = 1.59's, 67.920'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 38.439'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3418.165'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.286'MB/sec write SST data time = 0.13's, 283.380'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 31461.518'MB/sec zip my value time = 1.59's, unzip length = 0.108'GB zip my value throughput = 67.920'MB/sec zip pipeline throughput = 135.989'MB/sec entries = 936569 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.31 avg-zval = 32.21 usrkeys = 936569 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.31 avg-zval = 32.21 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 936569 bytes = 3395072 } seq expand size = 6555983 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5140 value = 3.5804 dict = 1.00 all = 3.6850 } Zip/UnZip{ index = 0.0951 value = 0.2793 dict = 1.00 all = 0.2714 } ---------------------------- total value len = 1.031080 GB avg = 0.123 KB (by entry num) total key len = 0.234132 GB avg = 0.028 KB total ukey len = 0.167237 GB avg = 0.020 KB total ukey num = 0.008361857 Billion total entry num = 0.008361857 Billion write speed all = 0.016353223 MB/sec (with seq num) write speed all = 0.015488588 MB/sec (without seq num) 2024/11/23-06:21:50.888732 140025129498176 [compaction_job.cc:1948] [default] [JOB 20937] Generated table #248347: 936569 keys, 36422572 bytes, temperature: kWarm 2024/11/23-06:21:50.888797 140025129498176 EVENT_LOG_v1 {"time_micros": 1732314110888761, "cf_name": "default", "job": 20937, "event": "table_file_creation", "file_number": 248347, "file_size": 36422572, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 3832926275, "table_properties": {"data_size": 30162672, "index_size": 1781568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26223932, "raw_average_key_size": 28, "raw_value_size": 107993930, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 936569, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732314107, "oldest_key_time": 0, "file_creation_time": 1732314107, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248347, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:21:50.923053 140025129498176 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248348.sst) /dev/shm/ToplingTemp/Topling-433003-eMNACP 2024/11/23-06:21:51.719204 140025129498176 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248348.sst): first pass time = 0.80's, 168.592'MB/sec 2024/11/23-06:21:53.264942 140025129498176 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248348.sst): old prealloc_size = 147639500, real_size = 30917968 2024/11/23-06:21:53.265518 140025129498176 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248348.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924958 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0289 GB avg-val = 117.11 avg-zval = 31.26 2024/11/23-06:21:53.266307 140025129498176 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248348.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:21:54.117835 140025129498176 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248348.sst): second pass time = 1.54's, 70.137'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 41.184'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3169.147'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.319'MB/sec write SST data time = 0.13's, 274.157'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31630.252'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.137'MB/sec zip pipeline throughput = 120.107'MB/sec entries = 924958 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.11 avg-zval = 31.26 usrkeys = 924958 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.11 avg-zval = 31.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924958 bytes = 3352992 } seq expand size = 6474706 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.4456 value = 3.7460 dict = 1.00 all = 3.8212 } Zip/UnZip{ index = 0.0957 value = 0.2669 dict = 1.00 all = 0.2617 } ---------------------------- total value len = 1.145874 GB avg = 0.123 KB (by entry num) total key len = 0.260031 GB avg = 0.028 KB total ukey len = 0.185736 GB avg = 0.020 KB total ukey num = 0.009286815 Billion total entry num = 0.009286815 Billion write speed all = 0.018170946 MB/sec (with seq num) write speed all = 0.017210709 MB/sec (without seq num) 2024/11/23-06:21:54.159401 140025129498176 [compaction_job.cc:1948] [default] [JOB 20937] Generated table #248348: 924958 keys, 35124786 bytes, temperature: kWarm 2024/11/23-06:21:54.159464 140025129498176 EVENT_LOG_v1 {"time_micros": 1732314114159433, "cf_name": "default", "job": 20937, "event": "table_file_creation", "file_number": 248348, "file_size": 35124786, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514179572, "largest_seqno": 3832925986, "table_properties": {"data_size": 28915520, "index_size": 1771008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25898824, "raw_average_key_size": 28, "raw_value_size": 108318964, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924958, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732314110, "oldest_key_time": 0, "file_creation_time": 1732314110, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248348, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:21:54.198752 140025129498176 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248349.sst) /dev/shm/ToplingTemp/Topling-433003-vMv4tS 2024/11/23-06:21:54.997818 140025129498176 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248349.sst): first pass time = 0.80's, 167.993'MB/sec 2024/11/23-06:21:56.530952 140025129498176 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248349.sst): old prealloc_size = 147639500, real_size = 32232384 2024/11/23-06:21:56.531540 140025129498176 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248349.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931928 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0302 GB avg-val = 116.02 avg-zval = 32.42 2024/11/23-06:21:56.532342 140025129498176 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248349.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:21:57.439829 140025129498176 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248349.sst): second pass time = 1.53's, 70.559'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 40.118'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3228.320'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.299'MB/sec write SST data time = 0.13's, 282.766'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33841.342'MB/sec zip my value time = 1.53's, unzip length = 0.108'GB zip my value throughput = 70.559'MB/sec zip pipeline throughput = 93.870'MB/sec entries = 931928 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.02 avg-zval = 32.42 usrkeys = 931928 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.02 avg-zval = 32.42 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931928 bytes = 3378256 } seq expand size = 6523496 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.4446 value = 3.5785 dict = 1.00 all = 3.6811 } Zip/UnZip{ index = 0.0957 value = 0.2794 dict = 1.00 all = 0.2717 } ---------------------------- total value len = 1.260521 GB avg = 0.123 KB (by entry num) total key len = 0.286125 GB avg = 0.028 KB total ukey len = 0.204375 GB avg = 0.020 KB total ukey num = 0.010218743 Billion total entry num = 0.010218743 Billion write speed all = 0.019989132 MB/sec (with seq num) write speed all = 0.018932581 MB/sec (without seq num) 2024/11/23-06:21:57.474668 140025129498176 [compaction_job.cc:1948] [default] [JOB 20937] Generated table #248349: 931928 keys, 36461080 bytes, temperature: kWarm 2024/11/23-06:21:57.474730 140025129498176 EVENT_LOG_v1 {"time_micros": 1732314117474698, "cf_name": "default", "job": 20937, "event": "table_file_creation", "file_number": 248349, "file_size": 36461080, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3485447251, "largest_seqno": 3832925997, "table_properties": {"data_size": 30214704, "index_size": 1784512, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26093984, "raw_average_key_size": 28, "raw_value_size": 108123820, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931928, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732314114, "oldest_key_time": 0, "file_creation_time": 1732314114, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248349, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:21:57.514717 140025129498176 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248350.sst) /dev/shm/ToplingTemp/Topling-433003-qxMObP 2024/11/23-06:21:58.366374 140025129498176 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248350.sst): first pass time = 0.85's, 157.604'MB/sec 2024/11/23-06:21:59.945317 140025129498176 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248350.sst): old prealloc_size = 147639500, real_size = 32574352 2024/11/23-06:21:59.945914 140025129498176 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248350.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928490 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0306 GB avg-val = 116.55 avg-zval = 32.92 2024/11/23-06:21:59.946696 140025129498176 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248350.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:22:00.865165 140025129498176 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248350.sst): second pass time = 1.58's, 68.592'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.000'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3085.041'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.293'MB/sec write SST data time = 0.13's, 287.235'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 28209.789'MB/sec zip my value time = 1.58's, unzip length = 0.108'GB zip my value throughput = 68.592'MB/sec zip pipeline throughput = 120.527'MB/sec entries = 928490 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.55 avg-zval = 32.92 usrkeys = 928490 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.55 avg-zval = 32.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928490 bytes = 3365792 } seq expand size = 6499430 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.4439 value = 3.5408 dict = 1.00 all = 3.6479 } Zip/UnZip{ index = 0.0957 value = 0.2824 dict = 1.00 all = 0.2741 } ---------------------------- total value len = 1.375241 GB avg = 0.123 KB (by entry num) total key len = 0.312123 GB avg = 0.028 KB total ukey len = 0.222945 GB avg = 0.020 KB total ukey num = 0.011147233 Billion total entry num = 0.011147233 Billion write speed all = 0.021806823 MB/sec (with seq num) write speed all = 0.020654323 MB/sec (without seq num) 2024/11/23-06:22:00.896224 140025129498176 [compaction_job.cc:1948] [default] [JOB 20937] Generated table #248350: 928490 keys, 36793312 bytes, temperature: kWarm 2024/11/23-06:22:00.896288 140025129498176 EVENT_LOG_v1 {"time_micros": 1732314120896257, "cf_name": "default", "job": 20937, "event": "table_file_creation", "file_number": 248350, "file_size": 36793312, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 3832926334, "table_properties": {"data_size": 30563968, "index_size": 1778048, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25997720, "raw_average_key_size": 28, "raw_value_size": 108220132, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928490, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732314117, "oldest_key_time": 0, "file_creation_time": 1732314117, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248350, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:22:00.933583 140025129498176 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248351.sst) /dev/shm/ToplingTemp/Topling-433003-GFaF4S 2024/11/23-06:22:01.037374 140025129498176 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248351.sst): first pass time = 0.10's, 228.860'MB/sec 2024/11/23-06:22:01.259007 140025129498176 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248351.sst): old prealloc_size = 147639500, real_size = 6364048 2024/11/23-06:22:01.259153 140025129498176 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248351.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 167046 prefix = 4 raw-key = 0.0027 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.0191 GB zip-val = 0.0060 GB avg-val = 114.14 avg-zval = 35.91 2024/11/23-06:22:01.259327 140025129498176 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248351.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0004 GB, waited 0.000 sec, Key+Value = 0.024 GB 2024/11/23-06:22:01.417318 140025129498176 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00/248351.sst): second pass time = 0.22's, 86.248'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 44.640'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2577.839'MB/sec (index lex order gen) rebuild zvType time = 0.13's, 0.311'MB/sec write SST data time = 0.02's, 297.763'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.200'MB, throughput = 7245.647'MB/sec zip my value time = 0.22's, unzip length = 0.019'GB zip my value throughput = 86.248'MB/sec zip pipeline throughput = 86.248'MB/sec entries = 167046 avg-key = 28.00 avg-zkey = 1.94 avg-val = 114.14 avg-zval = 35.91 usrkeys = 167046 avg-key = 20.00 avg-zkey = 1.94 avg-val = 121.14 avg-zval = 35.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 167046 bytes = 542912 } seq expand size = 1169322 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0033 GB value = 0.0191 GB dict = 0.20 MB all = 0.0237 GB } __ZipSize{ index = 0.0003 GB value = 0.0060 GB dict = 0.20 MB all = 0.0071 GB } UnZip/Zip{ index = 10.3329 value = 3.1785 dict = 1.00 all = 3.3601 } Zip/UnZip{ index = 0.0968 value = 0.3146 dict = 1.00 all = 0.2976 } ---------------------------- total value len = 1.395477 GB avg = 0.123 KB (by entry num) total key len = 0.316800 GB avg = 0.028 KB total ukey len = 0.226286 GB avg = 0.020 KB total ukey num = 0.011314279 Billion total entry num = 0.011314279 Billion write speed all = 0.022128641 MB/sec (with seq num) write speed all = 0.020958879 MB/sec (without seq num) 2024/11/23-06:22:01.430318 140025129498176 [compaction_job.cc:1948] [default] [JOB 20937] Generated table #248351: 167046 keys, 7066515 bytes, temperature: kWarm 2024/11/23-06:22:01.430377 140025129498176 EVENT_LOG_v1 {"time_micros": 1732314121430346, "cf_name": "default", "job": 20937, "event": "table_file_creation", "file_number": 248351, "file_size": 7066515, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3774281145, "largest_seqno": 3831776165, "table_properties": {"data_size": 5998784, "index_size": 323328, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4677288, "raw_average_key_size": 28, "raw_value_size": 19067196, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 167046, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732314120, "oldest_key_time": 0, "file_creation_time": 1732314120, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248351, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:22:01.442892 140025129498176 [compaction_job.cc:703] job-20937: subcompact[0], size: 0.444770 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20937/att-00 [248339,248340,248341,248342,248343,248344,248345,248346,248347,248348,248349,248350,248351] 2024/11/23-06:22:01.531082 140025129498176 (Original Log Time 2024/11/23-06:21:18.398979) EVENT_LOG_v1 {"time_micros": 1732314078398918, "job": 20937, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [248333], "files_L3": [248244, 248245, 248247, 248248, 248249, 248268, 248269, 248270, 248271, 248272, 248273, 248274], "score": 0.987235, "input_data_size": 468540348, "oldest_snapshot_seqno": 3832953130} 2024/11/23-06:22:01.531085 140025129498176 (Original Log Time 2024/11/23-06:22:01.530381) [compaction_job.cc:2022] [default] [JOB 20937] Compacted 1@2 + 12@3 files to L3 => 444770417 bytes 2024/11/23-06:22:01.531087 140025129498176 (Original Log Time 2024/11/23-06:22:01.531017) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 13 0 0 0] max score 0.02, MB/sec: 10.9 rd, 10.3 wr, level 3, files in(1, 12) out(13 +0 blob) MB in(27.1, 419.7 +0.0 blob) out(424.2 +0.0 blob), read-write-amplify(32.1) write-amplify(15.7) OK, records in: 11383060, records dropped: 68781 output_compression: Snappy 2024/11/23-06:22:01.531093 140025129498176 (Original Log Time 2024/11/23-06:22:01.531061) EVENT_LOG_v1 {"time_micros": 1732314121531041, "cf": "default", "job": 20937, "event": "compaction_finished", "compaction_time_micros": 43043919, "compaction_time_cpu_micros": 19118314, "output_level": 3, "num_output_files": 13, "total_output_size": 444770417, "num_input_records": 11383060, "num_output_records": 11314279, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 308139759, "file_range_sync_nanos": 0, "file_fsync_nanos": 12349, "file_prepare_write_nanos": 27691, "lsm_state": [0, 0, 0, 13, 0, 0, 0]} 2024/11/23-06:22:01.660927 140025129498176 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732314078,"executesMs":43301,"compactionJobId":20937,"attempt":0,"compactionInputRawBytes":1644125012,"compactionInputZipBytes":468540348,"compactionOutputRawBytes":0,"compactionOutputZipBytes":444770417,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-7ee02845a5468be2a31a1d87fdb05ddc-7ae815bc87a83630-00"}}, response = 2024/11/23-06:22:01.661013 140025129498176 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-20937/att-00: olev 3, work 43.301 s, result 6.324 ms, install 0.729 ms, input{raw 1.531 GiB zip 446.835 MiB}