2024/11/23-17:44:06.623695 139957542479424 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00: fork to child time = 0.008322 sec 2024/11/23-17:44:06.624473 139957542479424 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00 2024/11/23-17:44:06.632674 139957542479424 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00 2024/11/23-17:44:06.633576 139957542479424 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-17:44:06.633817 139957542479424 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-17:44:06.690508 139957542479424 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-32245/att-00/MANIFEST-206347 2024/11/23-17:44:06.706594 139957542479424 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-32245/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 272982, last_sequence is 4157341393, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 272844 2024/11/23-17:44:06.706601 139957542479424 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-17:44:06.706728 139957542479424 [version_set.cc:5721] Creating manifest 272982 2024/11/23-17:44:06.738963 139957542479424 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-17:44:06.763152 139957542479424 [compaction_job.cc:2437] [default] [JOB 32245] Compacting 2@2 + 22@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-17:44:06.763174 139957542479424 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 24135 Base level 2, inputs: [272978(31MB) 272977(31MB)], [272908(40MB) 272909(40MB) 272910(40MB) 272911(40MB) 272912(40MB) 272913(40MB) 272914(40MB) 272915(40MB) 272916(40MB) 272917(40MB) 272918(40MB) 272919(40MB) 272920(40MB) 272921(40MB) 272922(40MB) 272923(40MB) 272924(40MB) 272925(40MB) 272926(40MB) 272927(40MB) 272928(40MB) 272929(40MB)] 2024/11/23-17:44:06.772142 139957542479424 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:44:06.772174 139957542479424 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:44:06.772194 139957542479424 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:44:06.772200 139957542479424 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-17:44:06.772212 139957542479424 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-17:44:07.463935 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272983.sst) /dev/shm/ToplingTemp/Topling-137525-NRnKdd 2024/11/23-17:44:07.663343 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272983.sst): first pass time = 0.20's, 673.227'MB/sec 2024/11/23-17:44:08.722661 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272983.sst): old prealloc_size = 147639500, real_size = 40584544 2024/11/23-17:44:08.722818 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272983.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.99 avg-zval = 311.51 2024/11/23-17:44:08.723078 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272983.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:44:08.941515 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272983.sst): second pass time = 1.06's, 124.351'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.22's, 193.814'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1835.398'MB/sec (index lex order gen) rebuild zvType time = 0.17's, 0.193'MB/sec write SST data time = 0.05's, 837.042'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45444.393'MB/sec zip my value time = 1.06's, unzip length = 0.132'GB zip my value throughput = 124.351'MB/sec zip pipeline throughput = 142.495'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1016.99 avg-zval = 311.51 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1023.99 avg-zval = 311.51 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129431 bytes = 485376 } seq expand size = 906017 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6507 value = 3.2647 dict = 1.00 all = 3.1642 } Zip/UnZip{ index = 0.1504 value = 0.3063 dict = 1.00 all = 0.3160 } ---------------------------- total value len = 8.981155 GB avg = 0.171 KB (by entry num) total key len = 1.443410 GB avg = 0.028 KB total ukey len = 0.999431 GB avg = 0.015 KB total ukey num = 0.065679866 Billion total entry num = 0.052487595 Billion write speed all = 0.328470073 MB/sec (with seq num) write speed all = 0.315239321 MB/sec (without seq num) 2024/11/23-17:44:08.997802 139957542479424 [compaction_job.cc:1948] [default] [JOB 32245] Generated table #272983: 129431 keys, 42417169 bytes, temperature: kWarm 2024/11/23-17:44:08.997889 139957542479424 EVENT_LOG_v1 {"time_micros": 1732355048997833, "cf_name": "default", "job": 32245, "event": "table_file_creation", "file_number": 272983, "file_size": 42417169, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237877277, "largest_seqno": 4157314334, "table_properties": {"data_size": 40318480, "index_size": 233536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629484, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355047, "oldest_key_time": 0, "file_creation_time": 1732355047, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272983, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:44:09.052064 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272984.sst) /dev/shm/ToplingTemp/Topling-137525-Kw9Myc 2024/11/23-17:44:09.981121 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272984.sst): first pass time = 0.93's, 144.475'MB/sec 2024/11/23-17:44:11.459567 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272984.sst): old prealloc_size = 147639500, real_size = 40875328 2024/11/23-17:44:11.459719 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272984.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129433 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.74 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1016.97 avg-zval = 313.81 2024/11/23-17:44:11.459919 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272984.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:44:11.635738 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272984.sst): second pass time = 1.48's, 89.076'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 242.438'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1770.163'MB/sec (index lex order gen) rebuild zvType time = 0.13's, 0.256'MB/sec write SST data time = 0.05's, 858.915'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 39007.728'MB/sec zip my value time = 1.48's, unzip length = 0.132'GB zip my value throughput = 89.076'MB/sec zip pipeline throughput = 141.359'MB/sec entries = 129433 avg-key = 20.00 avg-zkey = 1.74 avg-val = 1016.97 avg-zval = 313.81 usrkeys = 129433 avg-key = 12.00 avg-zkey = 1.74 avg-val = 1023.97 avg-zval = 313.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129433 bytes = 485392 } seq expand size = 906031 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.9083 value = 3.2407 dict = 1.00 all = 3.1427 } Zip/UnZip{ index = 0.1448 value = 0.3086 dict = 1.00 all = 0.3182 } ---------------------------- total value len = 9.113691 GB avg = 0.173 KB (by entry num) total key len = 1.445998 GB avg = 0.027 KB total ukey len = 1.000984 GB avg = 0.015 KB total ukey num = 0.065809299 Billion total entry num = 0.052617028 Billion write speed all = 0.332699508 MB/sec (with seq num) write speed all = 0.319437255 MB/sec (without seq num) 2024/11/23-17:44:11.695371 139957542479424 [compaction_job.cc:1948] [default] [JOB 32245] Generated table #272984: 129433 keys, 42708059 bytes, temperature: kWarm 2024/11/23-17:44:11.695436 139957542479424 EVENT_LOG_v1 {"time_micros": 1732355051695402, "cf_name": "default", "job": 32245, "event": "table_file_creation", "file_number": 272984, "file_size": 42708059, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868195, "largest_seqno": 4157307756, "table_properties": {"data_size": 40617968, "index_size": 224832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588660, "raw_average_key_size": 20, "raw_value_size": 131630097, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129433, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355049, "oldest_key_time": 0, "file_creation_time": 1732355049, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272984, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:44:11.736805 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272985.sst) /dev/shm/ToplingTemp/Topling-137525-JSemaa 2024/11/23-17:44:12.303050 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272985.sst): first pass time = 0.57's, 237.047'MB/sec 2024/11/23-17:44:13.158633 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272985.sst): old prealloc_size = 147639500, real_size = 40579520 2024/11/23-17:44:13.158776 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272985.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129433 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.97 avg-zval = 311.46 2024/11/23-17:44:13.158927 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272985.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:44:13.336897 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272985.sst): second pass time = 0.85's, 154.286'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 237.978'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1895.877'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.269'MB/sec write SST data time = 0.06's, 733.976'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44284.456'MB/sec zip my value time = 0.85's, unzip length = 0.132'GB zip my value throughput = 154.286'MB/sec zip pipeline throughput = 154.286'MB/sec entries = 129433 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1016.97 avg-zval = 311.46 usrkeys = 129433 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1023.97 avg-zval = 311.46 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129433 bytes = 485392 } seq expand size = 906031 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6508 value = 3.2651 dict = 1.00 all = 3.1646 } Zip/UnZip{ index = 0.1504 value = 0.3063 dict = 1.00 all = 0.3160 } ---------------------------- total value len = 9.246227 GB avg = 0.175 KB (by entry num) total key len = 1.448587 GB avg = 0.027 KB total ukey len = 1.002538 GB avg = 0.015 KB total ukey num = 0.065938732 Billion total entry num = 0.052746461 Billion write speed all = 0.336938741 MB/sec (with seq num) write speed all = 0.323644577 MB/sec (without seq num) 2024/11/23-17:44:13.395327 139957542479424 [compaction_job.cc:1948] [default] [JOB 32245] Generated table #272985: 129433 keys, 42412081 bytes, temperature: kWarm 2024/11/23-17:44:13.395392 139957542479424 EVENT_LOG_v1 {"time_micros": 1732355053395354, "cf_name": "default", "job": 32245, "event": "table_file_creation", "file_number": 272985, "file_size": 42412081, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237873554, "largest_seqno": 4157318301, "table_properties": {"data_size": 40313456, "index_size": 233536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588660, "raw_average_key_size": 20, "raw_value_size": 131629302, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129433, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355051, "oldest_key_time": 0, "file_creation_time": 1732355051, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272985, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:44:13.432817 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272986.sst) /dev/shm/ToplingTemp/Topling-137525-0x9ira 2024/11/23-17:44:14.104883 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272986.sst): first pass time = 0.67's, 199.718'MB/sec 2024/11/23-17:44:15.019149 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272986.sst): old prealloc_size = 147639500, real_size = 40867632 2024/11/23-17:44:15.019297 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272986.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129434 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.74 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1016.96 avg-zval = 313.75 2024/11/23-17:44:15.019490 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272986.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:44:15.289121 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272986.sst): second pass time = 0.91's, 144.291'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.27's, 158.189'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1827.145'MB/sec (index lex order gen) rebuild zvType time = 0.22's, 0.145'MB/sec write SST data time = 0.05's, 913.262'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 46416.414'MB/sec zip my value time = 0.91's, unzip length = 0.132'GB zip my value throughput = 144.291'MB/sec zip pipeline throughput = 145.096'MB/sec entries = 129434 avg-key = 20.00 avg-zkey = 1.74 avg-val = 1016.96 avg-zval = 313.75 usrkeys = 129434 avg-key = 12.00 avg-zkey = 1.74 avg-val = 1023.96 avg-zval = 313.75 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129434 bytes = 485392 } seq expand size = 906038 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.9083 value = 3.2413 dict = 1.00 all = 3.1432 } Zip/UnZip{ index = 0.1448 value = 0.3085 dict = 1.00 all = 0.3181 } ---------------------------- total value len = 9.378762 GB avg = 0.177 KB (by entry num) total key len = 1.451176 GB avg = 0.027 KB total ukey len = 1.004091 GB avg = 0.015 KB total ukey num = 0.066068166 Billion total entry num = 0.052875895 Billion write speed all = 0.341174824 MB/sec (with seq num) write speed all = 0.327848857 MB/sec (without seq num) 2024/11/23-17:44:15.344721 139957542479424 [compaction_job.cc:1948] [default] [JOB 32245] Generated table #272986: 129434 keys, 42700720 bytes, temperature: kWarm 2024/11/23-17:44:15.344790 139957542479424 EVENT_LOG_v1 {"time_micros": 1732355055344752, "cf_name": "default", "job": 32245, "event": "table_file_creation", "file_number": 272986, "file_size": 42700720, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868596, "largest_seqno": 4157303041, "table_properties": {"data_size": 40610272, "index_size": 224832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588680, "raw_average_key_size": 20, "raw_value_size": 131629337, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129434, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355053, "oldest_key_time": 0, "file_creation_time": 1732355053, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272986, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:44:15.383494 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272987.sst) /dev/shm/ToplingTemp/Topling-137525-kGo8xa 2024/11/23-17:44:15.938996 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272987.sst): first pass time = 0.56's, 241.631'MB/sec 2024/11/23-17:44:16.771570 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272987.sst): old prealloc_size = 147639500, real_size = 40590416 2024/11/23-17:44:16.771689 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272987.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.99 avg-zval = 311.55 2024/11/23-17:44:16.771841 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272987.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:44:16.935521 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272987.sst): second pass time = 0.83's, 158.398'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 258.790'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2386.332'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.271'MB/sec write SST data time = 0.04's, 952.288'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 35820.488'MB/sec zip my value time = 0.83's, unzip length = 0.132'GB zip my value throughput = 158.398'MB/sec zip pipeline throughput = 158.398'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1016.99 avg-zval = 311.55 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1023.99 avg-zval = 311.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6506 value = 3.2643 dict = 1.00 all = 3.1638 } Zip/UnZip{ index = 0.1504 value = 0.3063 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 9.511298 GB avg = 0.179 KB (by entry num) total key len = 1.453764 GB avg = 0.027 KB total ukey len = 1.005644 GB avg = 0.015 KB total ukey num = 0.066197596 Billion total entry num = 0.053005325 Billion write speed all = 0.345413718 MB/sec (with seq num) write speed all = 0.332055824 MB/sec (without seq num) 2024/11/23-17:44:16.990481 139957542479424 [compaction_job.cc:1948] [default] [JOB 32245] Generated table #272987: 129430 keys, 42423233 bytes, temperature: kWarm 2024/11/23-17:44:16.990549 139957542479424 EVENT_LOG_v1 {"time_micros": 1732355056990514, "cf_name": "default", "job": 32245, "event": "table_file_creation", "file_number": 272987, "file_size": 42423233, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237869923, "largest_seqno": 4157316853, "table_properties": {"data_size": 40324352, "index_size": 233536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629616, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355055, "oldest_key_time": 0, "file_creation_time": 1732355055, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272987, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:44:17.028573 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272988.sst) /dev/shm/ToplingTemp/Topling-137525-xEwEic 2024/11/23-17:44:18.036550 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272988.sst): first pass time = 1.01's, 133.162'MB/sec 2024/11/23-17:44:19.112396 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272988.sst): old prealloc_size = 147639500, real_size = 40582608 2024/11/23-17:44:19.112523 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272988.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129432 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.98 avg-zval = 311.49 2024/11/23-17:44:19.112683 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272988.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:44:19.273869 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272988.sst): second pass time = 1.07's, 122.513'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 262.700'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2227.908'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.272'MB/sec write SST data time = 0.04's, 996.221'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 47341.063'MB/sec zip my value time = 1.07's, unzip length = 0.132'GB zip my value throughput = 122.513'MB/sec zip pipeline throughput = 127.980'MB/sec entries = 129432 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1016.98 avg-zval = 311.49 usrkeys = 129432 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1023.98 avg-zval = 311.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129432 bytes = 485392 } seq expand size = 906024 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6507 value = 3.2649 dict = 1.00 all = 3.1644 } Zip/UnZip{ index = 0.1504 value = 0.3063 dict = 1.00 all = 0.3160 } ---------------------------- total value len = 9.643834 GB avg = 0.181 KB (by entry num) total key len = 1.456353 GB avg = 0.027 KB total ukey len = 1.007197 GB avg = 0.015 KB total ukey num = 0.066327028 Billion total entry num = 0.053134757 Billion write speed all = 0.349644558 MB/sec (with seq num) write speed all = 0.336255033 MB/sec (without seq num) 2024/11/23-17:44:19.326522 139957542479424 [compaction_job.cc:1948] [default] [JOB 32245] Generated table #272988: 129432 keys, 42415426 bytes, temperature: kWarm 2024/11/23-17:44:19.326588 139957542479424 EVENT_LOG_v1 {"time_micros": 1732355059326553, "cf_name": "default", "job": 32245, "event": "table_file_creation", "file_number": 272988, "file_size": 42415426, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237872175, "largest_seqno": 4157317373, "table_properties": {"data_size": 40316544, "index_size": 233536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588640, "raw_average_key_size": 20, "raw_value_size": 131629782, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355057, "oldest_key_time": 0, "file_creation_time": 1732355057, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272988, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:44:19.366787 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272989.sst) /dev/shm/ToplingTemp/Topling-137525-xGsh4b 2024/11/23-17:44:20.012983 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272989.sst): first pass time = 0.65's, 207.719'MB/sec 2024/11/23-17:44:20.889150 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272989.sst): old prealloc_size = 147639500, real_size = 41066432 2024/11/23-17:44:20.889276 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272989.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0408 GB avg-val =1017.01 avg-zval = 315.32 2024/11/23-17:44:20.889448 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272989.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:44:21.055169 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272989.sst): second pass time = 0.88's, 150.369'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 258.432'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2162.754'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.273'MB/sec write SST data time = 0.05's, 902.441'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44320.974'MB/sec zip my value time = 0.88's, unzip length = 0.132'GB zip my value throughput = 150.369'MB/sec zip pipeline throughput = 150.369'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.01 avg-zval = 315.32 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1024.01 avg-zval = 315.32 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129427 bytes = 485360 } seq expand size = 905989 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0408 GB dict = 1.38 MB all = 0.0429 GB } UnZip/Zip{ index = 6.9694 value = 3.2253 dict = 1.00 all = 3.1287 } Zip/UnZip{ index = 0.1435 value = 0.3100 dict = 1.00 all = 0.3196 } ---------------------------- total value len = 9.776369 GB avg = 0.184 KB (by entry num) total key len = 1.458941 GB avg = 0.027 KB total ukey len = 1.008750 GB avg = 0.015 KB total ukey num = 0.066456455 Billion total entry num = 0.053264184 Billion write speed all = 0.353880962 MB/sec (with seq num) write speed all = 0.340459575 MB/sec (without seq num) 2024/11/23-17:44:21.107097 139957542479424 [compaction_job.cc:1948] [default] [JOB 32245] Generated table #272989: 129427 keys, 42899356 bytes, temperature: kWarm 2024/11/23-17:44:21.107163 139957542479424 EVENT_LOG_v1 {"time_micros": 1732355061107128, "cf_name": "default", "job": 32245, "event": "table_file_creation", "file_number": 272989, "file_size": 42899356, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237946515, "largest_seqno": 4157318386, "table_properties": {"data_size": 40811056, "index_size": 222848, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629189, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355059, "oldest_key_time": 0, "file_creation_time": 1732355059, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272989, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:44:21.146860 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272990.sst) /dev/shm/ToplingTemp/Topling-137525-GnsRDd 2024/11/23-17:44:22.168742 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272990.sst): first pass time = 1.02's, 131.350'MB/sec 2024/11/23-17:44:23.623075 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272990.sst): old prealloc_size = 147639500, real_size = 40904976 2024/11/23-17:44:23.623215 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272990.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.73 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1016.99 avg-zval = 314.05 2024/11/23-17:44:23.623402 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272990.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:44:23.821605 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272990.sst): second pass time = 1.45's, 90.590'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.20's, 215.315'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1935.644'MB/sec (index lex order gen) rebuild zvType time = 0.13's, 0.241'MB/sec write SST data time = 0.06's, 669.450'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 41651.566'MB/sec zip my value time = 1.45's, unzip length = 0.132'GB zip my value throughput = 90.590'MB/sec zip pipeline throughput = 98.781'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.73 avg-val = 1016.99 avg-zval = 314.05 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.73 avg-val = 1023.99 avg-zval = 314.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129431 bytes = 485376 } seq expand size = 906017 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.9259 value = 3.2383 dict = 1.00 all = 3.1405 } Zip/UnZip{ index = 0.1444 value = 0.3088 dict = 1.00 all = 0.3184 } ---------------------------- total value len = 9.908905 GB avg = 0.186 KB (by entry num) total key len = 1.461530 GB avg = 0.027 KB total ukey len = 1.010303 GB avg = 0.015 KB total ukey num = 0.066585886 Billion total entry num = 0.053393615 Billion write speed all = 0.358105813 MB/sec (with seq num) write speed all = 0.344652984 MB/sec (without seq num) 2024/11/23-17:44:23.876480 139957542479424 [compaction_job.cc:1948] [default] [JOB 32245] Generated table #272990: 129431 keys, 42737910 bytes, temperature: kWarm 2024/11/23-17:44:23.876570 139957542479424 EVENT_LOG_v1 {"time_micros": 1732355063876521, "cf_name": "default", "job": 32245, "event": "table_file_creation", "file_number": 272990, "file_size": 42737910, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868290, "largest_seqno": 4157315543, "table_properties": {"data_size": 40648192, "index_size": 224256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131630143, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355061, "oldest_key_time": 0, "file_creation_time": 1732355061, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272990, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:44:23.958696 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272991.sst) /dev/shm/ToplingTemp/Topling-137525-eY4oVa 2024/11/23-17:44:24.587577 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272991.sst): first pass time = 0.63's, 213.443'MB/sec 2024/11/23-17:44:25.463731 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272991.sst): old prealloc_size = 147639500, real_size = 40985664 2024/11/23-17:44:25.463854 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272991.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0407 GB avg-val =1017.00 avg-zval = 314.69 2024/11/23-17:44:25.464022 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272991.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:44:25.628855 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272991.sst): second pass time = 0.88's, 150.349'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 259.339'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2176.816'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.05's, 908.119'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 41773.166'MB/sec zip my value time = 0.88's, unzip length = 0.132'GB zip my value throughput = 150.349'MB/sec zip pipeline throughput = 150.349'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.00 avg-zval = 314.69 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1024.00 avg-zval = 314.69 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0407 GB dict = 1.38 MB all = 0.0428 GB } UnZip/Zip{ index = 6.9616 value = 3.2318 dict = 1.00 all = 3.1346 } Zip/UnZip{ index = 0.1436 value = 0.3094 dict = 1.00 all = 0.3190 } ---------------------------- total value len = 10.041441 GB avg = 0.188 KB (by entry num) total key len = 1.464119 GB avg = 0.027 KB total ukey len = 1.011857 GB avg = 0.015 KB total ukey num = 0.066715316 Billion total entry num = 0.053523045 Billion write speed all = 0.362340864 MB/sec (with seq num) write speed all = 0.348856192 MB/sec (without seq num) 2024/11/23-17:44:25.686192 139957542479424 [compaction_job.cc:1948] [default] [JOB 32245] Generated table #272991: 129430 keys, 42818126 bytes, temperature: kWarm 2024/11/23-17:44:25.686258 139957542479424 EVENT_LOG_v1 {"time_micros": 1732355065686220, "cf_name": "default", "job": 32245, "event": "table_file_creation", "file_number": 272991, "file_size": 42818126, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237919999, "largest_seqno": 4157318571, "table_properties": {"data_size": 40730032, "index_size": 223104, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629925, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355063, "oldest_key_time": 0, "file_creation_time": 1732355063, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272991, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:44:25.724843 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272992.sst) /dev/shm/ToplingTemp/Topling-137525-q8g8zd 2024/11/23-17:44:26.616286 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272992.sst): first pass time = 0.89's, 150.570'MB/sec 2024/11/23-17:44:27.436442 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272992.sst): old prealloc_size = 147639500, real_size = 40585696 2024/11/23-17:44:27.436566 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272992.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129433 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.97 avg-zval = 311.51 2024/11/23-17:44:27.436720 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272992.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:44:27.598389 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272992.sst): second pass time = 0.82's, 160.606'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 261.968'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2312.398'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.273'MB/sec write SST data time = 0.04's, 977.933'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44125.468'MB/sec zip my value time = 0.82's, unzip length = 0.132'GB zip my value throughput = 160.606'MB/sec zip pipeline throughput = 160.606'MB/sec entries = 129433 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1016.97 avg-zval = 311.51 usrkeys = 129433 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1023.97 avg-zval = 311.51 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129433 bytes = 485392 } seq expand size = 906031 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6508 value = 3.2646 dict = 1.00 all = 3.1642 } Zip/UnZip{ index = 0.1504 value = 0.3063 dict = 1.00 all = 0.3160 } ---------------------------- total value len = 10.173976 GB avg = 0.190 KB (by entry num) total key len = 1.466707 GB avg = 0.027 KB total ukey len = 1.013410 GB avg = 0.015 KB total ukey num = 0.066844749 Billion total entry num = 0.053652478 Billion write speed all = 0.366573547 MB/sec (with seq num) write speed all = 0.353057104 MB/sec (without seq num) 2024/11/23-17:44:27.652220 139957542479424 [compaction_job.cc:1948] [default] [JOB 32245] Generated table #272992: 129433 keys, 42418000 bytes, temperature: kWarm 2024/11/23-17:44:27.652289 139957542479424 EVENT_LOG_v1 {"time_micros": 1732355067652255, "cf_name": "default", "job": 32245, "event": "table_file_creation", "file_number": 272992, "file_size": 42418000, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237873877, "largest_seqno": 4157313832, "table_properties": {"data_size": 40319632, "index_size": 233536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588660, "raw_average_key_size": 20, "raw_value_size": 131629425, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129433, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355065, "oldest_key_time": 0, "file_creation_time": 1732355065, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272992, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:44:27.692603 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272993.sst) /dev/shm/ToplingTemp/Topling-137525-sXbFVd 2024/11/23-17:44:28.229733 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272993.sst): first pass time = 0.54's, 249.901'MB/sec 2024/11/23-17:44:29.008028 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272993.sst): old prealloc_size = 147639500, real_size = 40588624 2024/11/23-17:44:29.008159 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272993.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.00 avg-zval = 311.54 2024/11/23-17:44:29.008329 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272993.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:44:29.172051 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272993.sst): second pass time = 0.78's, 169.296'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 258.643'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2149.830'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.270'MB/sec write SST data time = 0.04's, 965.518'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 49384.458'MB/sec zip my value time = 0.78's, unzip length = 0.132'GB zip my value throughput = 169.296'MB/sec zip pipeline throughput = 169.296'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1017.00 avg-zval = 311.54 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1024.00 avg-zval = 311.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129429 bytes = 485376 } seq expand size = 906003 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6506 value = 3.2644 dict = 1.00 all = 3.1639 } Zip/UnZip{ index = 0.1504 value = 0.3063 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 10.306512 GB avg = 0.192 KB (by entry num) total key len = 1.469296 GB avg = 0.027 KB total ukey len = 1.014963 GB avg = 0.015 KB total ukey num = 0.066974178 Billion total entry num = 0.053781907 Billion write speed all = 0.370810333 MB/sec (with seq num) write speed all = 0.357261955 MB/sec (without seq num) 2024/11/23-17:44:29.225664 139957542479424 [compaction_job.cc:1948] [default] [JOB 32245] Generated table #272993: 129429 keys, 42421168 bytes, temperature: kWarm 2024/11/23-17:44:29.225732 139957542479424 EVENT_LOG_v1 {"time_micros": 1732355069225696, "cf_name": "default", "job": 32245, "event": "table_file_creation", "file_number": 272993, "file_size": 42421168, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237873605, "largest_seqno": 4157317412, "table_properties": {"data_size": 40322560, "index_size": 233536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629720, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355067, "oldest_key_time": 0, "file_creation_time": 1732355067, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272993, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:44:29.266020 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272994.sst) /dev/shm/ToplingTemp/Topling-137525-fdMEW9 2024/11/23-17:44:29.800374 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272994.sst): first pass time = 0.53's, 251.192'MB/sec 2024/11/23-17:44:30.695635 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272994.sst): old prealloc_size = 147639500, real_size = 40877792 2024/11/23-17:44:30.695763 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272994.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.76 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1016.99 avg-zval = 313.81 2024/11/23-17:44:30.695923 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272994.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:44:30.857191 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272994.sst): second pass time = 0.89's, 147.327'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 264.397'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2210.062'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.04's, 971.891'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45236.175'MB/sec zip my value time = 0.89's, unzip length = 0.132'GB zip my value throughput = 147.327'MB/sec zip pipeline throughput = 147.327'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.76 avg-val = 1016.99 avg-zval = 313.81 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.76 avg-val = 1023.99 avg-zval = 313.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129431 bytes = 485376 } seq expand size = 906017 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.8112 value = 3.2407 dict = 1.00 all = 3.1425 } Zip/UnZip{ index = 0.1468 value = 0.3086 dict = 1.00 all = 0.3182 } ---------------------------- total value len = 10.439047 GB avg = 0.194 KB (by entry num) total key len = 1.471885 GB avg = 0.027 KB total ukey len = 1.016516 GB avg = 0.015 KB total ukey num = 0.067103609 Billion total entry num = 0.053911338 Billion write speed all = 0.375045376 MB/sec (with seq num) write speed all = 0.361465113 MB/sec (without seq num) 2024/11/23-17:44:30.910127 139957542479424 [compaction_job.cc:1948] [default] [JOB 32245] Generated table #272994: 129431 keys, 42710075 bytes, temperature: kWarm 2024/11/23-17:44:30.910194 139957542479424 EVENT_LOG_v1 {"time_micros": 1732355070910160, "cf_name": "default", "job": 32245, "event": "table_file_creation", "file_number": 272994, "file_size": 42710075, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237872082, "largest_seqno": 4157308725, "table_properties": {"data_size": 40617232, "index_size": 228032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629437, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355069, "oldest_key_time": 0, "file_creation_time": 1732355069, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272994, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:44:30.946860 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272995.sst) /dev/shm/ToplingTemp/Topling-137525-7WHwOa 2024/11/23-17:44:31.449652 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272995.sst): first pass time = 0.50's, 266.967'MB/sec 2024/11/23-17:44:32.458712 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272995.sst): old prealloc_size = 147639500, real_size = 40874304 2024/11/23-17:44:32.458836 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272995.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.76 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1017.02 avg-zval = 313.80 2024/11/23-17:44:32.459008 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272995.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:44:32.621014 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272995.sst): second pass time = 1.01's, 130.542'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 263.200'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2215.882'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.273'MB/sec write SST data time = 0.04's, 977.999'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 37441.963'MB/sec zip my value time = 1.01's, unzip length = 0.132'GB zip my value throughput = 130.542'MB/sec zip pipeline throughput = 166.256'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.76 avg-val = 1017.02 avg-zval = 313.80 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.76 avg-val = 1024.02 avg-zval = 313.80 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129427 bytes = 485360 } seq expand size = 905989 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.8110 value = 3.2410 dict = 1.00 all = 3.1428 } Zip/UnZip{ index = 0.1468 value = 0.3085 dict = 1.00 all = 0.3182 } ---------------------------- total value len = 10.571583 GB avg = 0.196 KB (by entry num) total key len = 1.474473 GB avg = 0.027 KB total ukey len = 1.018069 GB avg = 0.015 KB total ukey num = 0.067233036 Billion total entry num = 0.054040765 Billion write speed all = 0.379279044 MB/sec (with seq num) write speed all = 0.365666935 MB/sec (without seq num) 2024/11/23-17:44:32.672753 139957542479424 [compaction_job.cc:1948] [default] [JOB 32245] Generated table #272995: 129427 keys, 42707037 bytes, temperature: kWarm 2024/11/23-17:44:32.672820 139957542479424 EVENT_LOG_v1 {"time_micros": 1732355072672787, "cf_name": "default", "job": 32245, "event": "table_file_creation", "file_number": 272995, "file_size": 42707037, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237871365, "largest_seqno": 4157316157, "table_properties": {"data_size": 40613744, "index_size": 228032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629997, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355070, "oldest_key_time": 0, "file_creation_time": 1732355070, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272995, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:44:32.710530 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272996.sst) /dev/shm/ToplingTemp/Topling-137525-t4Zjkb 2024/11/23-17:44:33.216724 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272996.sst): first pass time = 0.51's, 265.172'MB/sec 2024/11/23-17:44:34.044177 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272996.sst): old prealloc_size = 147639500, real_size = 40560816 2024/11/23-17:44:34.044303 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272996.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.76 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.01 avg-zval = 311.37 2024/11/23-17:44:34.044456 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272996.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:44:34.209929 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272996.sst): second pass time = 0.83's, 159.226'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 254.978'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2240.009'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.268'MB/sec write SST data time = 0.04's, 943.457'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 48585.484'MB/sec zip my value time = 0.83's, unzip length = 0.132'GB zip my value throughput = 159.226'MB/sec zip pipeline throughput = 159.226'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.76 avg-val = 1017.01 avg-zval = 311.37 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.76 avg-val = 1021.73 avg-zval = 311.37 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 87279 bytes = 327312 } seq expand size = 610953 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0423 GB } UnZip/Zip{ index = 6.7996 value = 3.2662 dict = 1.00 all = 3.1763 } Zip/UnZip{ index = 0.1471 value = 0.3062 dict = 1.00 all = 0.3148 } ---------------------------- total value len = 10.703824 GB avg = 0.198 KB (by entry num) total key len = 1.477062 GB avg = 0.027 KB total ukey len = 1.019622 GB avg = 0.015 KB total ukey num = 0.067362464 Billion total entry num = 0.054170193 Billion write speed all = 0.383505047 MB/sec (with seq num) write speed all = 0.369861019 MB/sec (without seq num) 2024/11/23-17:44:34.262063 139957542479424 [compaction_job.cc:1948] [default] [JOB 32245] Generated table #272996: 129428 keys, 42256096 bytes, temperature: kWarm 2024/11/23-17:44:34.262129 139957542479424 EVENT_LOG_v1 {"time_micros": 1732355074262094, "cf_name": "default", "job": 32245, "event": "table_file_creation", "file_number": 272996, "file_size": 42256096, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4157318149, "table_properties": {"data_size": 40299872, "index_size": 228416, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629409, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355072, "oldest_key_time": 0, "file_creation_time": 1732355072, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272996, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:44:34.297599 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272997.sst) /dev/shm/ToplingTemp/Topling-137525-L1vhkc 2024/11/23-17:44:34.978834 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272997.sst): first pass time = 0.68's, 197.032'MB/sec 2024/11/23-17:44:35.856457 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272997.sst): old prealloc_size = 147639500, real_size = 40786304 2024/11/23-17:44:35.856606 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272997.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129433 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.70 raw-val = 0.1316 GB zip-val = 0.0405 GB avg-val =1016.97 avg-zval = 313.16 2024/11/23-17:44:35.856744 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272997.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:44:36.058554 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272997.sst): second pass time = 0.88's, 150.339'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.20's, 209.882'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1750.987'MB/sec (index lex order gen) rebuild zvType time = 0.16's, 0.207'MB/sec write SST data time = 0.05's, 930.965'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44604.124'MB/sec zip my value time = 0.88's, unzip length = 0.132'GB zip my value throughput = 150.339'MB/sec zip pipeline throughput = 164.812'MB/sec entries = 129433 avg-key = 20.00 avg-zkey = 1.70 avg-val = 1016.97 avg-zval = 313.16 usrkeys = 129433 avg-key = 12.00 avg-zkey = 1.70 avg-val = 1020.71 avg-zval = 313.16 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 69228 bytes = 259616 } seq expand size = 484596 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0405 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 7.0467 value = 3.2474 dict = 1.00 all = 3.1645 } Zip/UnZip{ index = 0.1419 value = 0.3079 dict = 1.00 all = 0.3160 } ---------------------------- total value len = 10.835938 GB avg = 0.200 KB (by entry num) total key len = 1.479650 GB avg = 0.027 KB total ukey len = 1.021176 GB avg = 0.015 KB total ukey num = 0.067491897 Billion total entry num = 0.054299626 Billion write speed all = 0.387723484 MB/sec (with seq num) write speed all = 0.374047652 MB/sec (without seq num) 2024/11/23-17:44:36.109432 139957542479424 [compaction_job.cc:1948] [default] [JOB 32245] Generated table #272997: 129433 keys, 42413736 bytes, temperature: kWarm 2024/11/23-17:44:36.109501 139957542479424 EVENT_LOG_v1 {"time_micros": 1732355076109465, "cf_name": "default", "job": 32245, "event": "table_file_creation", "file_number": 272997, "file_size": 42413736, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4157309400, "table_properties": {"data_size": 40533360, "index_size": 220416, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588660, "raw_average_key_size": 20, "raw_value_size": 131629499, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129433, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355074, "oldest_key_time": 0, "file_creation_time": 1732355074, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272997, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:44:36.178898 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272998.sst) /dev/shm/ToplingTemp/Topling-137525-DykdT9 2024/11/23-17:44:36.804391 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272998.sst): first pass time = 0.63's, 214.594'MB/sec 2024/11/23-17:44:37.583211 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272998.sst): old prealloc_size = 147639500, real_size = 40809920 2024/11/23-17:44:37.583335 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272998.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.73 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1017.02 avg-zval = 313.33 2024/11/23-17:44:37.583491 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272998.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:44:37.748857 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272998.sst): second pass time = 0.78's, 169.190'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 256.614'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2238.090'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.268'MB/sec write SST data time = 0.04's, 948.660'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45278.820'MB/sec zip my value time = 0.78's, unzip length = 0.132'GB zip my value throughput = 169.190'MB/sec zip pipeline throughput = 169.190'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.73 avg-val = 1017.02 avg-zval = 313.33 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.73 avg-val = 1021.73 avg-zval = 313.33 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 87109 bytes = 326672 } seq expand size = 609763 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0425 GB } UnZip/Zip{ index = 6.9515 value = 3.2458 dict = 1.00 all = 3.1577 } Zip/UnZip{ index = 0.1439 value = 0.3081 dict = 1.00 all = 0.3167 } ---------------------------- total value len = 10.968177 GB avg = 0.202 KB (by entry num) total key len = 1.482239 GB avg = 0.027 KB total ukey len = 1.022729 GB avg = 0.015 KB total ukey num = 0.067621324 Billion total entry num = 0.054429053 Billion write speed all = 0.391947329 MB/sec (with seq num) write speed all = 0.378239629 MB/sec (without seq num) 2024/11/23-17:44:37.801252 139957542479424 [compaction_job.cc:1948] [default] [JOB 32245] Generated table #272998: 129427 keys, 42504513 bytes, temperature: kWarm 2024/11/23-17:44:37.801319 139957542479424 EVENT_LOG_v1 {"time_micros": 1732355077801283, "cf_name": "default", "job": 32245, "event": "table_file_creation", "file_number": 272998, "file_size": 42504513, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4157309145, "table_properties": {"data_size": 40553968, "index_size": 223424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629758, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355076, "oldest_key_time": 0, "file_creation_time": 1732355076, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272998, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:44:37.842815 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272999.sst) /dev/shm/ToplingTemp/Topling-137525-zJ7Mfe 2024/11/23-17:44:38.365890 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272999.sst): first pass time = 0.52's, 256.613'MB/sec 2024/11/23-17:44:39.198141 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272999.sst): old prealloc_size = 147639500, real_size = 41064736 2024/11/23-17:44:39.198265 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272999.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129426 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.71 raw-val = 0.1316 GB zip-val = 0.0408 GB avg-val =1017.03 avg-zval = 315.32 2024/11/23-17:44:39.198437 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272999.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:44:39.391222 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/272999.sst): second pass time = 0.83's, 158.471'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.19's, 222.145'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2235.805'MB/sec (index lex order gen) rebuild zvType time = 0.13's, 0.255'MB/sec write SST data time = 0.07's, 650.565'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 51053.718'MB/sec zip my value time = 0.83's, unzip length = 0.132'GB zip my value throughput = 158.471'MB/sec zip pipeline throughput = 158.471'MB/sec entries = 129426 avg-key = 20.00 avg-zkey = 1.71 avg-val = 1017.03 avg-zval = 315.32 usrkeys = 129426 avg-key = 12.00 avg-zkey = 1.71 avg-val = 1023.52 avg-zval = 315.32 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 120000 bytes = 450016 } seq expand size = 840000 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0408 GB dict = 1.38 MB all = 0.0429 GB } UnZip/Zip{ index = 6.9995 value = 3.2254 dict = 1.00 all = 3.1299 } Zip/UnZip{ index = 0.1429 value = 0.3100 dict = 1.00 all = 0.3195 } ---------------------------- total value len = 11.100647 GB avg = 0.203 KB (by entry num) total key len = 1.484827 GB avg = 0.027 KB total ukey len = 1.024282 GB avg = 0.015 KB total ukey num = 0.067750750 Billion total entry num = 0.054558479 Billion write speed all = 0.396178569 MB/sec (with seq num) write speed all = 0.382438984 MB/sec (without seq num) 2024/11/23-17:44:39.446735 139957542479424 [compaction_job.cc:1948] [default] [JOB 32245] Generated table #272999: 129426 keys, 42882540 bytes, temperature: kWarm 2024/11/23-17:44:39.446828 139957542479424 EVENT_LOG_v1 {"time_micros": 1732355079446778, "cf_name": "default", "job": 32245, "event": "table_file_creation", "file_number": 272999, "file_size": 42882540, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4157282093, "table_properties": {"data_size": 40810320, "index_size": 221888, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588520, "raw_average_key_size": 20, "raw_value_size": 131629746, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129426, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355077, "oldest_key_time": 0, "file_creation_time": 1732355077, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272999, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:44:39.496498 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273000.sst) /dev/shm/ToplingTemp/Topling-137525-5U8rib 2024/11/23-17:44:40.239570 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273000.sst): first pass time = 0.74's, 180.638'MB/sec 2024/11/23-17:44:41.010206 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273000.sst): old prealloc_size = 147639500, real_size = 40575456 2024/11/23-17:44:41.010328 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273000.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.77 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.00 avg-zval = 311.48 2024/11/23-17:44:41.010479 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273000.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:44:41.174455 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273000.sst): second pass time = 0.77's, 170.996'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 257.375'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2291.388'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.270'MB/sec write SST data time = 0.04's, 952.938'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 48056.044'MB/sec zip my value time = 0.77's, unzip length = 0.132'GB zip my value throughput = 170.996'MB/sec zip pipeline throughput = 170.996'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.77 avg-val = 1017.00 avg-zval = 311.48 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.77 avg-val = 1021.71 avg-zval = 311.48 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 87056 bytes = 326480 } seq expand size = 609392 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0423 GB } UnZip/Zip{ index = 6.7844 value = 3.2651 dict = 1.00 all = 3.1753 } Zip/UnZip{ index = 0.1474 value = 0.3063 dict = 1.00 all = 0.3149 } ---------------------------- total value len = 11.232886 GB avg = 0.205 KB (by entry num) total key len = 1.487416 GB avg = 0.027 KB total ukey len = 1.025835 GB avg = 0.015 KB total ukey num = 0.067880179 Billion total entry num = 0.054687908 Billion write speed all = 0.400400325 MB/sec (with seq num) write speed all = 0.386628918 MB/sec (without seq num) 2024/11/23-17:44:41.226144 139957542479424 [compaction_job.cc:1948] [default] [JOB 32245] Generated table #273000: 129429 keys, 42269670 bytes, temperature: kWarm 2024/11/23-17:44:41.226203 139957542479424 EVENT_LOG_v1 {"time_micros": 1732355081226171, "cf_name": "default", "job": 32245, "event": "table_file_creation", "file_number": 273000, "file_size": 42269670, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4157312165, "table_properties": {"data_size": 40314000, "index_size": 228928, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629304, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355079, "oldest_key_time": 0, "file_creation_time": 1732355079, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273000, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:44:41.261885 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273001.sst) /dev/shm/ToplingTemp/Topling-137525-NLTxua 2024/11/23-17:44:41.844959 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273001.sst): first pass time = 0.58's, 230.207'MB/sec 2024/11/23-17:44:42.614040 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273001.sst): old prealloc_size = 147639500, real_size = 40502160 2024/11/23-17:44:42.614173 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273001.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1017.01 avg-zval = 310.96 2024/11/23-17:44:42.614265 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273001.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:44:42.777797 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273001.sst): second pass time = 0.77's, 171.323'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 256.429'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2019.901'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.269'MB/sec write SST data time = 0.04's, 969.535'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45590.518'MB/sec zip my value time = 0.77's, unzip length = 0.132'GB zip my value throughput = 171.323'MB/sec zip pipeline throughput = 171.323'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.01 avg-zval = 310.96 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1018.70 avg-zval = 310.96 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 31287 bytes = 117344 } seq expand size = 219009 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0420 GB } UnZip/Zip{ index = 6.9936 value = 3.2705 dict = 1.00 all = 3.1967 } Zip/UnZip{ index = 0.1430 value = 0.3058 dict = 1.00 all = 0.3128 } ---------------------------- total value len = 11.364735 GB avg = 0.207 KB (by entry num) total key len = 1.490005 GB avg = 0.027 KB total ukey len = 1.027388 GB avg = 0.015 KB total ukey num = 0.068009608 Billion total entry num = 0.054817337 Billion write speed all = 0.404611637 MB/sec (with seq num) write speed all = 0.390808334 MB/sec (without seq num) 2024/11/23-17:44:42.829399 139957542479424 [compaction_job.cc:1948] [default] [JOB 32245] Generated table #273001: 129429 keys, 41987234 bytes, temperature: kWarm 2024/11/23-17:44:42.829471 139957542479424 EVENT_LOG_v1 {"time_micros": 1732355082829434, "cf_name": "default", "job": 32245, "event": "table_file_creation", "file_number": 273001, "file_size": 41987234, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4157316903, "table_properties": {"data_size": 40247552, "index_size": 222080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131630139, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355081, "oldest_key_time": 0, "file_creation_time": 1732355081, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273001, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:44:42.867479 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273002.sst) /dev/shm/ToplingTemp/Topling-137525-VPHL9b 2024/11/23-17:44:43.584440 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273002.sst): first pass time = 0.72's, 187.216'MB/sec 2024/11/23-17:44:44.417841 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273002.sst): old prealloc_size = 147639500, real_size = 40569584 2024/11/23-17:44:44.417969 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273002.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.02 avg-zval = 311.49 2024/11/23-17:44:44.418070 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273002.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:44:44.574653 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273002.sst): second pass time = 0.83's, 158.087'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 268.219'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1882.337'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.279'MB/sec write SST data time = 0.04's, 1031.035'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 42644.933'MB/sec zip my value time = 0.83's, unzip length = 0.132'GB zip my value throughput = 158.087'MB/sec zip pipeline throughput = 158.087'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.02 avg-zval = 311.49 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1018.70 avg-zval = 311.49 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 31067 bytes = 116512 } seq expand size = 217469 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0421 GB } UnZip/Zip{ index = 6.9935 value = 3.2650 dict = 1.00 all = 3.1916 } Zip/UnZip{ index = 0.1430 value = 0.3063 dict = 1.00 all = 0.3133 } ---------------------------- total value len = 11.496583 GB avg = 0.209 KB (by entry num) total key len = 1.492593 GB avg = 0.027 KB total ukey len = 1.028941 GB avg = 0.015 KB total ukey num = 0.068139035 Billion total entry num = 0.054946764 Billion write speed all = 0.408819985 MB/sec (with seq num) write speed all = 0.394984875 MB/sec (without seq num) 2024/11/23-17:44:44.625969 139957542479424 [compaction_job.cc:1948] [default] [JOB 32245] Generated table #273002: 129427 keys, 42053910 bytes, temperature: kWarm 2024/11/23-17:44:44.626042 139957542479424 EVENT_LOG_v1 {"time_micros": 1732355084626008, "cf_name": "default", "job": 32245, "event": "table_file_creation", "file_number": 273002, "file_size": 42053910, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4157295927, "table_properties": {"data_size": 40314976, "index_size": 222080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131630175, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355082, "oldest_key_time": 0, "file_creation_time": 1732355082, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273002, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:44:44.661938 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273003.sst) /dev/shm/ToplingTemp/Topling-137525-ew4zxa 2024/11/23-17:44:45.221225 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273003.sst): first pass time = 0.56's, 240.002'MB/sec 2024/11/23-17:44:46.018545 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273003.sst): old prealloc_size = 147639500, real_size = 40461536 2024/11/23-17:44:46.018670 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273003.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129432 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1016.97 avg-zval = 310.64 2024/11/23-17:44:46.018751 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273003.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:44:46.175692 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273003.sst): second pass time = 0.80's, 165.394'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 266.954'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2178.857'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.278'MB/sec write SST data time = 0.04's, 1027.248'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 41678.553'MB/sec zip my value time = 0.80's, unzip length = 0.132'GB zip my value throughput = 165.394'MB/sec zip pipeline throughput = 165.394'MB/sec entries = 129432 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1016.97 avg-zval = 310.64 usrkeys = 129432 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1018.62 avg-zval = 310.64 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 30439 bytes = 114160 } seq expand size = 213073 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0419 GB } UnZip/Zip{ index = 6.9938 value = 3.2738 dict = 1.00 all = 3.2000 } Zip/UnZip{ index = 0.1430 value = 0.3055 dict = 1.00 all = 0.3125 } ---------------------------- total value len = 11.628425 GB avg = 0.211 KB (by entry num) total key len = 1.495182 GB avg = 0.027 KB total ukey len = 1.030494 GB avg = 0.015 KB total ukey num = 0.068268467 Billion total entry num = 0.055076196 Billion write speed all = 0.413030234 MB/sec (with seq num) write speed all = 0.399163232 MB/sec (without seq num) 2024/11/23-17:44:46.225856 139957542479424 [compaction_job.cc:1948] [default] [JOB 32245] Generated table #273003: 129432 keys, 41943393 bytes, temperature: kWarm 2024/11/23-17:44:46.225925 139957542479424 EVENT_LOG_v1 {"time_micros": 1732355086225890, "cf_name": "default", "job": 32245, "event": "table_file_creation", "file_number": 273003, "file_size": 41943393, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4157314544, "table_properties": {"data_size": 40206928, "index_size": 222080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588640, "raw_average_key_size": 20, "raw_value_size": 131629092, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355084, "oldest_key_time": 0, "file_creation_time": 1732355084, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273003, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:44:46.265420 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273004.sst) /dev/shm/ToplingTemp/Topling-137525-X20vCa 2024/11/23-17:44:47.187321 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273004.sst): first pass time = 0.92's, 145.594'MB/sec 2024/11/23-17:44:48.017605 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273004.sst): old prealloc_size = 147639500, real_size = 41033904 2024/11/23-17:44:48.017725 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273004.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0408 GB avg-val =1016.98 avg-zval = 315.06 2024/11/23-17:44:48.017880 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273004.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:44:48.179307 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273004.sst): second pass time = 0.83's, 158.800'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 265.017'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2280.608'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.274'MB/sec write SST data time = 0.04's, 989.750'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 47197.767'MB/sec zip my value time = 0.83's, unzip length = 0.132'GB zip my value throughput = 158.800'MB/sec zip pipeline throughput = 158.800'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1016.98 avg-zval = 315.06 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1023.41 avg-zval = 315.06 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 118831 bytes = 445632 } seq expand size = 831817 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0408 GB dict = 1.38 MB all = 0.0428 GB } UnZip/Zip{ index = 6.9797 value = 3.2279 dict = 1.00 all = 3.1325 } Zip/UnZip{ index = 0.1433 value = 0.3098 dict = 1.00 all = 0.3192 } ---------------------------- total value len = 11.760886 GB avg = 0.213 KB (by entry num) total key len = 1.497770 GB avg = 0.027 KB total ukey len = 1.032048 GB avg = 0.015 KB total ukey num = 0.068397898 Billion total entry num = 0.055205627 Billion write speed all = 0.417254249 MB/sec (with seq num) write speed all = 0.403355536 MB/sec (without seq num) 2024/11/23-17:44:48.230648 139957542479424 [compaction_job.cc:1948] [default] [JOB 32245] Generated table #273004: 129431 keys, 42847233 bytes, temperature: kWarm 2024/11/23-17:44:48.230717 139957542479424 EVENT_LOG_v1 {"time_micros": 1732355088230682, "cf_name": "default", "job": 32245, "event": "table_file_creation", "file_number": 273004, "file_size": 42847233, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4157318367, "table_properties": {"data_size": 40778848, "index_size": 222528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629287, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355086, "oldest_key_time": 0, "file_creation_time": 1732355086, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273004, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:44:48.268858 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273005.sst) /dev/shm/ToplingTemp/Topling-137525-msPe1b 2024/11/23-17:44:48.279691 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273005.sst): first pass time = 0.01's, 411.996'MB/sec 2024/11/23-17:44:48.317111 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273005.sst): old prealloc_size = 147639500, real_size = 1408752 2024/11/23-17:44:48.317158 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273005.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 4284 prefix = 4 raw-key = 0.0000 GB zip-key = 0.0000 GB avg-key = 8.00 avg-zkey = 0.28 raw-val = 0.0044 GB zip-val = 0.0014 GB avg-val =1016.92 avg-zval = 328.27 2024/11/23-17:44:48.324688 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00/273005.sst): second pass time = 0.04's, 118.284'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.01's, 194.038'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 45.892'MB/sec (index lex order gen) rebuild zvType time = 0.01's, 0.181'MB/sec write SST data time = 0.00's, 907.721'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.046'MB, throughput = 1821.687'MB/sec zip my value time = 0.04's, unzip length = 0.004'GB zip my value throughput = 118.284'MB/sec zip pipeline throughput = 118.284'MB/sec entries = 4284 avg-key = 20.00 avg-zkey = 0.28 avg-val = 1016.92 avg-zval = 328.27 usrkeys = 4284 avg-key = 12.00 avg-zkey = 0.28 avg-val = 1023.92 avg-zval = 328.27 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 4284 bytes = 13408 } seq expand size = 29988 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0001 GB value = 0.0044 GB dict = 0.05 MB all = 0.0044 GB } __ZipSize{ index = 0.0000 GB value = 0.0014 GB dict = 0.05 MB all = 0.0015 GB } UnZip/Zip{ index = 42.2763 value = 3.0978 dict = 1.00 all = 3.0262 } Zip/UnZip{ index = 0.0237 value = 0.3228 dict = 1.00 all = 0.3305 } ---------------------------- total value len = 11.765272 GB avg = 0.213 KB (by entry num) total key len = 1.497856 GB avg = 0.027 KB total ukey len = 1.032099 GB avg = 0.015 KB total ukey num = 0.068402182 Billion total entry num = 0.055209911 Billion write speed all = 0.417393080 MB/sec (with seq num) write speed all = 0.403493352 MB/sec (without seq num) 2024/11/23-17:44:48.332742 139957542479424 [compaction_job.cc:1948] [default] [JOB 32245] Generated table #273005: 4284 keys, 1467924 bytes, temperature: kWarm 2024/11/23-17:44:48.332792 139957542479424 EVENT_LOG_v1 {"time_micros": 1732355088332763, "cf_name": "default", "job": 32245, "event": "table_file_creation", "file_number": 273005, "file_size": 1467924, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4137732296, "largest_seqno": 4155808749, "table_properties": {"data_size": 1406320, "index_size": 1216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 85680, "raw_average_key_size": 20, "raw_value_size": 4356505, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 4284, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1732355088, "oldest_key_time": 0, "file_creation_time": 1732355088, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273005, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:44:48.341768 139957542479424 [compaction_job.cc:703] job-32245: subcompact[0], size: 0.936415 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32245/att-00 [272983,272984,272985,272986,272987,272988,272989,272990,272991,272992,272993,272994,272995,272996,272997,272998,272999,273000,273001,273002,273003,273004,273005] 2024/11/23-17:44:48.481778 139957542479424 (Original Log Time 2024/11/23-17:44:06.763249) EVENT_LOG_v1 {"time_micros": 1732355046763191, "job": 32245, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [272978, 272977], "files_L3": [272908, 272909, 272910, 272911, 272912, 272913, 272914, 272915, 272916, 272917, 272918, 272919, 272920, 272921, 272922, 272923, 272924, 272925, 272926, 272927, 272928, 272929], "score": 0.986555, "input_data_size": 1000343310, "oldest_snapshot_seqno": 4157341383} 2024/11/23-17:44:48.481782 139957542479424 (Original Log Time 2024/11/23-17:44:48.480165) [compaction_job.cc:2022] [default] [JOB 32245] Compacted 2@2 + 22@3 files to L3 => 936414609 bytes 2024/11/23-17:44:48.481784 139957542479424 (Original Log Time 2024/11/23-17:44:48.481706) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 23 0 0 0] max score 0.03, MB/sec: 24.1 rd, 22.5 wr, level 3, files in(2, 22) out(23 +0 blob) MB in(62.4, 891.6 +0.0 blob) out(893.0 +0.0 blob), read-write-amplify(29.6) write-amplify(14.3) OK, records in: 2911299, records dropped: 59552 output_compression: Snappy 2024/11/23-17:44:48.481787 139957542479424 (Original Log Time 2024/11/23-17:44:48.481753) EVENT_LOG_v1 {"time_micros": 1732355088481731, "cf": "default", "job": 32245, "event": "compaction_finished", "compaction_time_micros": 41578523, "compaction_time_cpu_micros": 8611453, "output_level": 3, "num_output_files": 23, "total_output_size": 936414609, "num_input_records": 2911299, "num_output_records": 2851747, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 692040171, "file_range_sync_nanos": 0, "file_fsync_nanos": 17679, "file_prepare_write_nanos": 48421, "lsm_state": [0, 0, 0, 23, 0, 0, 0]} 2024/11/23-17:44:48.619437 139957542479424 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732355046,"executesMs":41848,"compactionJobId":32245,"attempt":0,"compactionInputRawBytes":3018997006,"compactionInputZipBytes":1000343310,"compactionOutputRawBytes":0,"compactionOutputZipBytes":936414609,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-31dad194b2ae85009bf147d9ac1cc674-37dd038c19a6e679-00"}}, response = 2024/11/23-17:44:48.619524 139957542479424 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-32245/att-00: olev 3, work 41.849 s, result 7.657 ms, install 1.639 ms, input{raw 2.812 GiB zip 954.002 MiB}