2024/11/23-17:05:18.053296 139957634799168 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00: fork to child time = 0.010393 sec 2024/11/23-17:05:18.054070 139957634799168 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00 2024/11/23-17:05:18.061423 139957634799168 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00 2024/11/23-17:05:18.062325 139957634799168 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-17:05:18.062548 139957634799168 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-17:05:18.117106 139957634799168 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-31470/att-00/MANIFEST-206347 2024/11/23-17:05:18.139519 139957634799168 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-31470/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 271266, last_sequence is 4133457935, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 271100 2024/11/23-17:05:18.139529 139957634799168 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-17:05:18.139674 139957634799168 [version_set.cc:5721] Creating manifest 271266 2024/11/23-17:05:18.174372 139957634799168 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-17:05:18.197157 139957634799168 [compaction_job.cc:2437] [default] [JOB 31470] Compacting 1@2 + 23@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-17:05:18.197172 139957634799168 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 23542 Base level 2, inputs: [271255(27MB)], [271252(37MB) 271253(897KB) 271144(37MB) 271188(38MB) 271189(38MB) 271190(34MB) 271191(34MB) 271192(35MB) 271193(34MB) 271194(35MB) 271195(37MB) 271196(35MB) 271197(37MB) 271198(34MB) 271199(35MB) 271200(37MB) 271201(35MB) 271202(37MB) 271203(35MB) 271204(35MB) 271205(36MB) 271206(37MB) 271260(37MB)] 2024/11/23-17:05:18.204617 139957634799168 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:05:18.204640 139957634799168 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:05:18.204655 139957634799168 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:05:18.204660 139957634799168 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-17:05:18.204670 139957634799168 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-17:05:18.795258 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271267.sst) /dev/shm/ToplingTemp/Topling-126049-aLg6oq 2024/11/23-17:05:19.532652 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271267.sst): first pass time = 0.74's, 182.030'MB/sec 2024/11/23-17:05:23.082888 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271267.sst): old prealloc_size = 147639500, real_size = 35037760 2024/11/23-17:05:23.083482 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271267.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925789 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0330 GB avg-val = 116.98 avg-zval = 35.69 2024/11/23-17:05:23.084229 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271267.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:05:24.169551 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271267.sst): second pass time = 3.55's, 30.516'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.09's, 35.800'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3089.552'MB/sec (index lex order gen) rebuild zvType time = 0.85's, 0.272'MB/sec write SST data time = 0.23's, 165.842'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30238.265'MB/sec zip my value time = 3.55's, unzip length = 0.108'GB zip my value throughput = 30.516'MB/sec zip pipeline throughput = 127.193'MB/sec entries = 925789 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.98 avg-zval = 35.69 usrkeys = 925789 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.98 avg-zval = 35.69 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925789 bytes = 3008832 } seq expand size = 6480523 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0330 GB dict = 1.08 MB all = 0.0389 GB } UnZip/Zip{ index = 10.5138 value = 3.2772 dict = 1.00 all = 3.4502 } Zip/UnZip{ index = 0.0951 value = 0.3051 dict = 1.00 all = 0.2898 } ---------------------------- total value len = 8.963396 GB avg = 0.168 KB (by entry num) total key len = 1.466743 GB avg = 0.028 KB total ukey len = 1.016394 GB avg = 0.015 KB total ukey num = 0.066476224 Billion total entry num = 0.053283953 Billion write speed all = 0.354622439 MB/sec (with seq num) write speed all = 0.340129297 MB/sec (without seq num) 2024/11/23-17:05:24.235165 139957634799168 [compaction_job.cc:1948] [default] [JOB 31470] Generated table #271267: 925789 keys, 38901270 bytes, temperature: kWarm 2024/11/23-17:05:24.235247 139957634799168 EVENT_LOG_v1 {"time_micros": 1732352724235195, "cf_name": "default", "job": 31470, "event": "table_file_creation", "file_number": 271267, "file_size": 38901270, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4094024658, "largest_seqno": 4133207057, "table_properties": {"data_size": 33045040, "index_size": 1761088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25922092, "raw_average_key_size": 28, "raw_value_size": 108295678, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925789, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352718, "oldest_key_time": 0, "file_creation_time": 1732352718, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271267, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:05:24.287905 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271268.sst) /dev/shm/ToplingTemp/Topling-126049-HuxS1r 2024/11/23-17:05:25.672304 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271268.sst): first pass time = 1.38's, 96.954'MB/sec 2024/11/23-17:05:27.772891 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271268.sst): old prealloc_size = 147639500, real_size = 35155728 2024/11/23-17:05:27.773472 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271268.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925845 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0332 GB avg-val = 116.97 avg-zval = 35.81 2024/11/23-17:05:27.774226 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271268.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:05:28.837767 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271268.sst): second pass time = 2.10's, 51.570'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.06's, 36.753'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3140.680'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.279'MB/sec write SST data time = 0.23's, 167.245'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33046.757'MB/sec zip my value time = 2.10's, unzip length = 0.108'GB zip my value throughput = 51.570'MB/sec zip pipeline throughput = 88.106'MB/sec entries = 925845 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.97 avg-zval = 35.81 usrkeys = 925845 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.97 avg-zval = 35.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925845 bytes = 3124736 } seq expand size = 6480915 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0391 GB } UnZip/Zip{ index = 10.4458 value = 3.2666 dict = 1.00 all = 3.4296 } Zip/UnZip{ index = 0.0957 value = 0.3061 dict = 1.00 all = 0.2916 } ---------------------------- total value len = 9.078171 GB avg = 0.167 KB (by entry num) total key len = 1.492667 GB avg = 0.028 KB total ukey len = 1.034911 GB avg = 0.015 KB total ukey num = 0.067402069 Billion total entry num = 0.054209798 Billion write speed all = 0.359349127 MB/sec (with seq num) write speed all = 0.344606497 MB/sec (without seq num) 2024/11/23-17:05:28.890705 139957634799168 [compaction_job.cc:1948] [default] [JOB 31470] Generated table #271268: 925845 keys, 39135526 bytes, temperature: kWarm 2024/11/23-17:05:28.890794 139957634799168 EVENT_LOG_v1 {"time_micros": 1732352728890746, "cf_name": "default", "job": 31470, "event": "table_file_creation", "file_number": 271268, "file_size": 39135526, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4051668904, "largest_seqno": 4133207102, "table_properties": {"data_size": 33151424, "index_size": 1772672, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25923660, "raw_average_key_size": 28, "raw_value_size": 108294082, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925845, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352724, "oldest_key_time": 0, "file_creation_time": 1732352724, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271268, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:05:28.943255 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271269.sst) /dev/shm/ToplingTemp/Topling-126049-afixNr 2024/11/23-17:05:30.161861 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271269.sst): first pass time = 1.22's, 110.145'MB/sec 2024/11/23-17:05:32.323849 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271269.sst): old prealloc_size = 147639500, real_size = 35376016 2024/11/23-17:05:32.324411 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271269.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928186 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0334 GB avg-val = 116.60 avg-zval = 35.96 2024/11/23-17:05:32.325215 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271269.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:05:33.398616 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271269.sst): second pass time = 2.16's, 50.083'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.07's, 36.629'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3240.398'MB/sec (index lex order gen) rebuild zvType time = 0.84's, 0.277'MB/sec write SST data time = 0.24's, 166.157'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33827.202'MB/sec zip my value time = 2.16's, unzip length = 0.108'GB zip my value throughput = 50.083'MB/sec zip pipeline throughput = 75.797'MB/sec entries = 928186 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.60 avg-zval = 35.96 usrkeys = 928186 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.60 avg-zval = 35.96 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928186 bytes = 3132640 } seq expand size = 6497302 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0334 GB dict = 1.08 MB all = 0.0394 GB } UnZip/Zip{ index = 10.5166 value = 3.2425 dict = 1.00 all = 3.4095 } Zip/UnZip{ index = 0.0951 value = 0.3084 dict = 1.00 all = 0.2933 } ---------------------------- total value len = 9.192897 GB avg = 0.167 KB (by entry num) total key len = 1.518656 GB avg = 0.028 KB total ukey len = 1.053474 GB avg = 0.015 KB total ukey num = 0.068330255 Billion total entry num = 0.055137984 Billion write speed all = 0.364076203 MB/sec (with seq num) write speed all = 0.349083473 MB/sec (without seq num) 2024/11/23-17:05:33.448937 139957634799168 [compaction_job.cc:1948] [default] [JOB 31470] Generated table #271269: 928186 keys, 39365586 bytes, temperature: kWarm 2024/11/23-17:05:33.449017 139957634799168 EVENT_LOG_v1 {"time_micros": 1732352733448973, "cf_name": "default", "job": 31470, "event": "table_file_creation", "file_number": 271269, "file_size": 39365586, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4003482220, "largest_seqno": 4133207166, "table_properties": {"data_size": 33378624, "index_size": 1765184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25989208, "raw_average_key_size": 28, "raw_value_size": 108228604, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928186, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352728, "oldest_key_time": 0, "file_creation_time": 1732352728, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271269, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:05:33.488250 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271270.sst) /dev/shm/ToplingTemp/Topling-126049-8X9w4p 2024/11/23-17:05:34.481705 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271270.sst): first pass time = 0.99's, 135.107'MB/sec 2024/11/23-17:05:36.106640 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271270.sst): old prealloc_size = 147639500, real_size = 36412880 2024/11/23-17:05:36.107195 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271270.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934790 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1080 GB zip-val = 0.0344 GB avg-val = 115.58 avg-zval = 36.79 2024/11/23-17:05:36.107946 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271270.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:05:37.102961 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271270.sst): second pass time = 1.62's, 66.550'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.00's, 40.595'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3340.667'MB/sec (index lex order gen) rebuild zvType time = 0.85's, 0.275'MB/sec write SST data time = 0.15's, 277.711'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 38107.049'MB/sec zip my value time = 1.62's, unzip length = 0.108'GB zip my value throughput = 66.550'MB/sec zip pipeline throughput = 66.550'MB/sec entries = 934790 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.58 avg-zval = 36.79 usrkeys = 934790 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.84 avg-zval = 36.79 TagRS{ kinds = RS_Key0_TagN bytes = 146160 } TagArray{ size = 836278 bytes = 3031520 } seq expand size = 5853946 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0404 GB } UnZip/Zip{ index = 10.4445 value = 3.1418 dict = 1.00 all = 3.3186 } Zip/UnZip{ index = 0.0957 value = 0.3183 dict = 1.00 all = 0.3013 } ---------------------------- total value len = 9.306795 GB avg = 0.166 KB (by entry num) total key len = 1.544830 GB avg = 0.028 KB total ukey len = 1.072170 GB avg = 0.015 KB total ukey num = 0.069265045 Billion total entry num = 0.056072774 Billion write speed all = 0.368790683 MB/sec (with seq num) write speed all = 0.353545691 MB/sec (without seq num) 2024/11/23-17:05:37.152905 139957634799168 [compaction_job.cc:1948] [default] [JOB 31470] Generated table #271270: 934790 keys, 40444376 bytes, temperature: kWarm 2024/11/23-17:05:37.153134 139957634799168 EVENT_LOG_v1 {"time_micros": 1732352737153096, "cf_name": "default", "job": 31470, "event": "table_file_creation", "file_number": 271270, "file_size": 40444376, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776165, "table_properties": {"data_size": 34388992, "index_size": 1790016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26174120, "raw_average_key_size": 28, "raw_value_size": 108043660, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934790, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352733, "oldest_key_time": 0, "file_creation_time": 1732352733, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271270, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:05:37.193756 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271271.sst) /dev/shm/ToplingTemp/Topling-126049-vkNydp 2024/11/23-17:05:38.223365 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271271.sst): first pass time = 1.03's, 130.363'MB/sec 2024/11/23-17:05:39.793779 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271271.sst): old prealloc_size = 147639500, real_size = 34995680 2024/11/23-17:05:39.794615 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271271.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1081665 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1039 GB zip-val = 0.0327 GB avg-val = 96.08 avg-zval = 30.20 2024/11/23-17:05:39.794673 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271271.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:05:40.703754 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271271.sst): second pass time = 1.57's, 66.217'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 39.310'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2534.954'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.363'MB/sec write SST data time = 0.17's, 216.593'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.035'MB, throughput = 32812.443'MB/sec zip my value time = 1.57's, unzip length = 0.104'GB zip my value throughput = 66.217'MB/sec zip pipeline throughput = 66.217'MB/sec entries = 1081665 avg-key = 28.00 avg-zkey = 1.90 avg-val = 96.08 avg-zval = 30.20 usrkeys = 1081665 avg-key = 20.00 avg-zkey = 1.90 avg-val = 96.08 avg-zval = 30.20 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0216 GB value = 0.1039 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0327 GB dict = 1.03 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5155 value = 3.1815 dict = 1.00 all = 3.7521 } Zip/UnZip{ index = 0.0951 value = 0.3143 dict = 1.00 all = 0.2665 } ---------------------------- total value len = 9.410726 GB avg = 0.165 KB (by entry num) total key len = 1.575117 GB avg = 0.028 KB total ukey len = 1.093804 GB avg = 0.016 KB total ukey num = 0.070346710 Billion total entry num = 0.057154439 Billion write speed all = 0.373306369 MB/sec (with seq num) write speed all = 0.357769197 MB/sec (without seq num) 2024/11/23-17:05:40.748547 139957634799168 [compaction_job.cc:1948] [default] [JOB 31470] Generated table #271271: 1081665 keys, 35770936 bytes, temperature: kWarm 2024/11/23-17:05:40.748596 139957634799168 EVENT_LOG_v1 {"time_micros": 1732352740748567, "cf_name": "default", "job": 31470, "event": "table_file_creation", "file_number": 271271, "file_size": 35770936, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 32667792, "index_size": 2057280, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30286620, "raw_average_key_size": 28, "raw_value_size": 103931135, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1081665, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352737, "oldest_key_time": 0, "file_creation_time": 1732352737, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271271, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:05:40.788731 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271272.sst) /dev/shm/ToplingTemp/Topling-126049-Jc6yWq 2024/11/23-17:05:41.736023 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271272.sst): first pass time = 0.95's, 141.691'MB/sec 2024/11/23-17:05:43.101330 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271272.sst): old prealloc_size = 147639500, real_size = 33623984 2024/11/23-17:05:43.101875 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271272.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922830 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0316 GB avg-val = 117.44 avg-zval = 34.28 2024/11/23-17:05:43.102597 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271272.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:05:43.918703 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271272.sst): second pass time = 1.36's, 79.464'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 45.647'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3353.127'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 291.790'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34405.198'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.464'MB/sec zip pipeline throughput = 79.464'MB/sec entries = 922830 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.44 avg-zval = 34.28 usrkeys = 922830 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.86 avg-zval = 34.28 TagRS{ kinds = RS_Key0_TagN bytes = 144280 } TagArray{ size = 714604 bytes = 2679776 } seq expand size = 5002228 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5138 value = 3.4256 dict = 1.00 all = 3.5974 } Zip/UnZip{ index = 0.0951 value = 0.2919 dict = 1.00 all = 0.2780 } ---------------------------- total value len = 9.524106 GB avg = 0.164 KB (by entry num) total key len = 1.600956 GB avg = 0.028 KB total ukey len = 1.112260 GB avg = 0.016 KB total ukey num = 0.071269540 Billion total entry num = 0.058077269 Billion write speed all = 0.377995866 MB/sec (with seq num) write speed all = 0.362209551 MB/sec (without seq num) 2024/11/23-17:05:43.965814 139957634799168 [compaction_job.cc:1948] [default] [JOB 31470] Generated table #271272: 922830 keys, 37309898 bytes, temperature: kWarm 2024/11/23-17:05:43.965888 139957634799168 EVENT_LOG_v1 {"time_micros": 1732352743965850, "cf_name": "default", "job": 31470, "event": "table_file_creation", "file_number": 271272, "file_size": 37309898, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012775885, "table_properties": {"data_size": 31637664, "index_size": 1755456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25839240, "raw_average_key_size": 28, "raw_value_size": 108378564, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922830, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352740, "oldest_key_time": 0, "file_creation_time": 1732352740, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271272, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:05:44.009091 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271273.sst) /dev/shm/ToplingTemp/Topling-126049-UxGO4o 2024/11/23-17:05:45.035613 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271273.sst): first pass time = 1.03's, 130.755'MB/sec 2024/11/23-17:05:46.347543 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271273.sst): old prealloc_size = 147639500, real_size = 32963616 2024/11/23-17:05:46.348111 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271273.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930646 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.22 avg-zval = 33.26 2024/11/23-17:05:46.348909 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271273.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:05:47.206351 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271273.sst): second pass time = 1.31's, 82.491'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.304'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3284.380'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 293.343'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34527.991'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.491'MB/sec zip pipeline throughput = 82.491'MB/sec entries = 930646 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.22 avg-zval = 33.26 usrkeys = 930646 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.22 avg-zval = 33.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930646 bytes = 3373600 } seq expand size = 6514522 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.4449 value = 3.4948 dict = 1.00 all = 3.6091 } Zip/UnZip{ index = 0.0957 value = 0.2861 dict = 1.00 all = 0.2771 } ---------------------------- total value len = 9.638781 GB avg = 0.163 KB (by entry num) total key len = 1.627014 GB avg = 0.028 KB total ukey len = 1.130873 GB avg = 0.016 KB total ukey num = 0.072200186 Billion total entry num = 0.059007915 Billion write speed all = 0.382734768 MB/sec (with seq num) write speed all = 0.366697280 MB/sec (without seq num) 2024/11/23-17:05:47.251375 139957634799168 [compaction_job.cc:1948] [default] [JOB 31470] Generated table #271273: 930646 keys, 37188454 bytes, temperature: kWarm 2024/11/23-17:05:47.251446 139957634799168 EVENT_LOG_v1 {"time_micros": 1732352747251411, "cf_name": "default", "job": 31470, "event": "table_file_creation", "file_number": 271273, "file_size": 37188454, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3487497056, "largest_seqno": 4012775902, "table_properties": {"data_size": 30948752, "index_size": 1782016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26058088, "raw_average_key_size": 28, "raw_value_size": 108159664, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930646, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352743, "oldest_key_time": 0, "file_creation_time": 1732352743, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271273, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:05:47.288355 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271274.sst) /dev/shm/ToplingTemp/Topling-126049-IKEYFq 2024/11/23-17:05:48.314737 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271274.sst): first pass time = 1.03's, 130.773'MB/sec 2024/11/23-17:05:49.630887 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271274.sst): old prealloc_size = 147639500, real_size = 32747200 2024/11/23-17:05:49.631469 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271274.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932240 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0307 GB avg-val = 115.97 avg-zval = 32.97 2024/11/23-17:05:49.632237 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271274.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:05:50.493790 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271274.sst): second pass time = 1.32's, 82.194'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.718'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3159.556'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.316'MB/sec write SST data time = 0.12's, 295.839'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35289.461'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.194'MB/sec zip pipeline throughput = 82.194'MB/sec entries = 932240 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.97 avg-zval = 32.97 usrkeys = 932240 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.97 avg-zval = 32.97 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932240 bytes = 3262848 } seq expand size = 6525680 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5118 value = 3.5170 dict = 1.00 all = 3.6412 } Zip/UnZip{ index = 0.0951 value = 0.2843 dict = 1.00 all = 0.2746 } ---------------------------- total value len = 9.753421 GB avg = 0.163 KB (by entry num) total key len = 1.653117 GB avg = 0.028 KB total ukey len = 1.149518 GB avg = 0.016 KB total ukey num = 0.073132426 Billion total entry num = 0.059940155 Billion write speed all = 0.387472995 MB/sec (with seq num) write speed all = 0.371183957 MB/sec (without seq num) 2024/11/23-17:05:50.539827 139957634799168 [compaction_job.cc:1948] [default] [JOB 31470] Generated table #271274: 932240 keys, 36860710 bytes, temperature: kWarm 2024/11/23-17:05:50.539897 139957634799168 EVENT_LOG_v1 {"time_micros": 1732352750539862, "cf_name": "default", "job": 31470, "event": "table_file_creation", "file_number": 271274, "file_size": 36860710, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3807172320, "largest_seqno": 4012775918, "table_properties": {"data_size": 30740272, "index_size": 1773696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26102720, "raw_average_key_size": 28, "raw_value_size": 108115060, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932240, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352747, "oldest_key_time": 0, "file_creation_time": 1732352747, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271274, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:05:50.583158 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271275.sst) /dev/shm/ToplingTemp/Topling-126049-an4ZWs 2024/11/23-17:05:51.609758 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271275.sst): first pass time = 1.03's, 130.745'MB/sec 2024/11/23-17:05:52.932356 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271275.sst): old prealloc_size = 147639500, real_size = 33126800 2024/11/23-17:05:52.932924 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271275.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929507 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.40 avg-zval = 33.49 2024/11/23-17:05:52.934867 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271275.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:05:53.792152 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271275.sst): second pass time = 1.32's, 81.878'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.575'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3257.401'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 295.011'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34757.674'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.878'MB/sec zip pipeline throughput = 81.878'MB/sec entries = 929507 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.40 avg-zval = 33.49 usrkeys = 929507 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.40 avg-zval = 33.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929507 bytes = 3485664 } seq expand size = 6506549 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5136 value = 3.4759 dict = 1.00 all = 3.5825 } Zip/UnZip{ index = 0.0951 value = 0.2877 dict = 1.00 all = 0.2791 } ---------------------------- total value len = 9.868120 GB avg = 0.162 KB (by entry num) total key len = 1.679143 GB avg = 0.028 KB total ukey len = 1.168108 GB avg = 0.016 KB total ukey num = 0.074061933 Billion total entry num = 0.060869662 Billion write speed all = 0.392209370 MB/sec (with seq num) write speed all = 0.375669587 MB/sec (without seq num) 2024/11/23-17:05:53.837038 139957634799168 [compaction_job.cc:1948] [default] [JOB 31470] Generated table #271275: 929507 keys, 37464924 bytes, temperature: kWarm 2024/11/23-17:05:53.837108 139957634799168 EVENT_LOG_v1 {"time_micros": 1732352753837073, "cf_name": "default", "job": 31470, "event": "table_file_creation", "file_number": 271275, "file_size": 37464924, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 4012774578, "table_properties": {"data_size": 31126016, "index_size": 1768192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26026196, "raw_average_key_size": 28, "raw_value_size": 108191630, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929507, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352750, "oldest_key_time": 0, "file_creation_time": 1732352750, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271275, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:05:53.875010 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271276.sst) /dev/shm/ToplingTemp/Topling-126049-lIcMrp 2024/11/23-17:05:54.817015 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271276.sst): first pass time = 0.94's, 142.486'MB/sec 2024/11/23-17:05:56.155832 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271276.sst): old prealloc_size = 147639500, real_size = 34670816 2024/11/23-17:05:56.156413 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271276.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928829 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0327 GB avg-val = 116.50 avg-zval = 35.16 2024/11/23-17:05:56.158000 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271276.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:05:57.000576 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271276.sst): second pass time = 1.34's, 80.874'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.040'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3174.542'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 308.801'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 24470.455'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.874'MB/sec zip pipeline throughput = 80.874'MB/sec entries = 928829 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.50 avg-zval = 35.16 usrkeys = 928829 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.50 avg-zval = 35.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928829 bytes = 3367024 } seq expand size = 6501803 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0327 GB dict = 1.08 MB all = 0.0389 GB } UnZip/Zip{ index = 10.4451 value = 3.3133 dict = 1.00 all = 3.4511 } Zip/UnZip{ index = 0.0957 value = 0.3018 dict = 1.00 all = 0.2898 } ---------------------------- total value len = 9.982832 GB avg = 0.162 KB (by entry num) total key len = 1.705150 GB avg = 0.028 KB total ukey len = 1.186685 GB avg = 0.016 KB total ukey num = 0.074990762 Billion total entry num = 0.061798491 Billion write speed all = 0.396945735 MB/sec (with seq num) write speed all = 0.380155396 MB/sec (without seq num) 2024/11/23-17:05:57.047661 139957634799168 [compaction_job.cc:1948] [default] [JOB 31470] Generated table #271276: 928829 keys, 38891084 bytes, temperature: kWarm 2024/11/23-17:05:57.047732 139957634799168 EVENT_LOG_v1 {"time_micros": 1732352757047697, "cf_name": "default", "job": 31470, "event": "table_file_creation", "file_number": 271276, "file_size": 38891084, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775967, "table_properties": {"data_size": 32659920, "index_size": 1778496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26007212, "raw_average_key_size": 28, "raw_value_size": 108210614, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928829, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352753, "oldest_key_time": 0, "file_creation_time": 1732352753, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271276, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:05:57.089140 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271277.sst) /dev/shm/ToplingTemp/Topling-126049-vMg1Vs 2024/11/23-17:05:58.042502 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271277.sst): first pass time = 0.95's, 140.789'MB/sec 2024/11/23-17:05:59.373075 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271277.sst): old prealloc_size = 147639500, real_size = 33467856 2024/11/23-17:05:59.373636 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271277.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927051 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 116.78 avg-zval = 33.95 2024/11/23-17:05:59.374440 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271277.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:06:00.216132 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271277.sst): second pass time = 1.33's, 81.415'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.699'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3257.550'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 299.327'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 23925.552'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.415'MB/sec zip pipeline throughput = 81.415'MB/sec entries = 927051 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.78 avg-zval = 33.95 usrkeys = 927051 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.78 avg-zval = 33.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927051 bytes = 3360576 } seq expand size = 6489357 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5148 value = 3.4398 dict = 1.00 all = 3.5618 } Zip/UnZip{ index = 0.0951 value = 0.2907 dict = 1.00 all = 0.2808 } ---------------------------- total value len = 10.097582 GB avg = 0.161 KB (by entry num) total key len = 1.731108 GB avg = 0.028 KB total ukey len = 1.205226 GB avg = 0.016 KB total ukey num = 0.075917813 Billion total entry num = 0.062725542 Billion write speed all = 0.401680546 MB/sec (with seq num) write speed all = 0.384640193 MB/sec (without seq num) 2024/11/23-17:06:00.261528 139957634799168 [compaction_job.cc:1948] [default] [JOB 31470] Generated table #271277: 927051 keys, 37682966 bytes, temperature: kWarm 2024/11/23-17:06:00.261597 139957634799168 EVENT_LOG_v1 {"time_micros": 1732352760261560, "cf_name": "default", "job": 31470, "event": "table_file_creation", "file_number": 271277, "file_size": 37682966, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 4012776021, "table_properties": {"data_size": 31472576, "index_size": 1763328, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25957428, "raw_average_key_size": 28, "raw_value_size": 108260346, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927051, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352757, "oldest_key_time": 0, "file_creation_time": 1732352757, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271277, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:06:00.297229 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271278.sst) /dev/shm/ToplingTemp/Topling-126049-r6JDVo 2024/11/23-17:06:01.351342 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271278.sst): first pass time = 1.05's, 127.332'MB/sec 2024/11/23-17:06:02.710555 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271278.sst): old prealloc_size = 147639500, real_size = 36024544 2024/11/23-17:06:02.711126 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271278.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924421 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0340 GB avg-val = 117.19 avg-zval = 36.80 2024/11/23-17:06:02.711837 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271278.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:06:03.523752 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271278.sst): second pass time = 1.36's, 79.754'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 48.956'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3220.674'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.337'MB/sec write SST data time = 0.13's, 312.854'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 33105.120'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.754'MB/sec zip pipeline throughput = 79.754'MB/sec entries = 924421 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.19 avg-zval = 36.80 usrkeys = 924421 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.81 avg-zval = 36.80 TagRS{ kinds = RS_Key0_TagN bytes = 144560 } TagArray{ size = 741424 bytes = 2780352 } seq expand size = 5189968 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0340 GB dict = 1.08 MB all = 0.0398 GB } UnZip/Zip{ index = 10.4444 value = 3.1841 dict = 1.00 all = 3.3715 } Zip/UnZip{ index = 0.0957 value = 0.3141 dict = 1.00 all = 0.2966 } ---------------------------- total value len = 10.211106 GB avg = 0.160 KB (by entry num) total key len = 1.756991 GB avg = 0.028 KB total ukey len = 1.223714 GB avg = 0.016 KB total ukey num = 0.076842234 Billion total entry num = 0.063649963 Billion write speed all = 0.406368934 MB/sec (with seq num) write speed all = 0.389079390 MB/sec (without seq num) 2024/11/23-17:06:03.571241 139957634799168 [compaction_job.cc:1948] [default] [JOB 31470] Generated table #271278: 924421 keys, 39809600 bytes, temperature: kWarm 2024/11/23-17:06:03.571313 139957634799168 EVENT_LOG_v1 {"time_micros": 1732352763571276, "cf_name": "default", "job": 31470, "event": "table_file_creation", "file_number": 271278, "file_size": 39809600, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012774830, "table_properties": {"data_size": 34023056, "index_size": 1770176, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25883788, "raw_average_key_size": 28, "raw_value_size": 108333998, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924421, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352760, "oldest_key_time": 0, "file_creation_time": 1732352760, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271278, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:06:03.608055 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271279.sst) /dev/shm/ToplingTemp/Topling-126049-ffPbHq 2024/11/23-17:06:04.443375 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271279.sst): first pass time = 0.84's, 160.685'MB/sec 2024/11/23-17:06:05.750032 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271279.sst): old prealloc_size = 147639500, real_size = 32548688 2024/11/23-17:06:05.750630 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271279.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932804 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0305 GB avg-val = 115.89 avg-zval = 32.74 2024/11/23-17:06:05.752187 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271279.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:06:06.619821 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271279.sst): second pass time = 1.31's, 82.778'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.286'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3108.355'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.13's, 284.343'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36109.287'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.778'MB/sec zip pipeline throughput = 82.778'MB/sec entries = 932804 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.89 avg-zval = 32.74 usrkeys = 932804 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.89 avg-zval = 32.74 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932804 bytes = 3381424 } seq expand size = 6529628 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5144 value = 3.5395 dict = 1.00 all = 3.6492 } Zip/UnZip{ index = 0.0951 value = 0.2825 dict = 1.00 all = 0.2740 } ---------------------------- total value len = 10.325735 GB avg = 0.160 KB (by entry num) total key len = 1.783110 GB avg = 0.028 KB total ukey len = 1.242370 GB avg = 0.016 KB total ukey num = 0.077775038 Billion total entry num = 0.064582767 Billion write speed all = 0.411104705 MB/sec (with seq num) write speed all = 0.393563623 MB/sec (without seq num) 2024/11/23-17:06:06.664831 139957634799168 [compaction_job.cc:1948] [default] [JOB 31470] Generated table #271279: 932804 keys, 36780526 bytes, temperature: kWarm 2024/11/23-17:06:06.664905 139957634799168 EVENT_LOG_v1 {"time_micros": 1732352766664868, "cf_name": "default", "job": 31470, "event": "table_file_creation", "file_number": 271279, "file_size": 36780526, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776228, "table_properties": {"data_size": 30540992, "index_size": 1774336, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26118512, "raw_average_key_size": 28, "raw_value_size": 108099340, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932804, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352763, "oldest_key_time": 0, "file_creation_time": 1732352763, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271279, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:06:06.702254 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271280.sst) /dev/shm/ToplingTemp/Topling-126049-KrZUqq 2024/11/23-17:06:07.637206 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271280.sst): first pass time = 0.93's, 143.562'MB/sec 2024/11/23-17:06:08.952659 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271280.sst): old prealloc_size = 147639500, real_size = 31957136 2024/11/23-17:06:08.953231 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271280.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 936365 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0299 GB avg-val = 115.34 avg-zval = 31.98 2024/11/23-17:06:08.954816 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271280.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:06:09.814765 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271280.sst): second pass time = 1.31's, 82.152'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 41.992'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3245.046'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.13's, 280.518'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 32894.666'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.152'MB/sec zip pipeline throughput = 82.152'MB/sec entries = 936365 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.34 avg-zval = 31.98 usrkeys = 936365 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.34 avg-zval = 31.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 936365 bytes = 3394336 } seq expand size = 6554555 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.5136 value = 3.6070 dict = 1.00 all = 3.7077 } Zip/UnZip{ index = 0.0951 value = 0.2772 dict = 1.00 all = 0.2697 } ---------------------------- total value len = 10.440289 GB avg = 0.159 KB (by entry num) total key len = 1.809328 GB avg = 0.028 KB total ukey len = 1.261097 GB avg = 0.016 KB total ukey num = 0.078711403 Billion total entry num = 0.065519132 Billion write speed all = 0.415838931 MB/sec (with seq num) write speed all = 0.398045457 MB/sec (without seq num) 2024/11/23-17:06:09.859620 139957634799168 [compaction_job.cc:1948] [default] [JOB 31470] Generated table #271280: 936365 keys, 36200054 bytes, temperature: kWarm 2024/11/23-17:06:09.859692 139957634799168 EVENT_LOG_v1 {"time_micros": 1732352769859655, "cf_name": "default", "job": 31470, "event": "table_file_creation", "file_number": 271280, "file_size": 36200054, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776248, "table_properties": {"data_size": 29941632, "index_size": 1781248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26218220, "raw_average_key_size": 28, "raw_value_size": 107999614, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 936365, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352766, "oldest_key_time": 0, "file_creation_time": 1732352766, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271280, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:06:09.895745 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271281.sst) /dev/shm/ToplingTemp/Topling-126049-BXZmLp 2024/11/23-17:06:10.949529 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271281.sst): first pass time = 1.05's, 127.372'MB/sec 2024/11/23-17:06:12.340444 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271281.sst): old prealloc_size = 147639500, real_size = 35006720 2024/11/23-17:06:12.341088 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271281.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925568 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0330 GB avg-val = 117.01 avg-zval = 35.66 2024/11/23-17:06:12.343796 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271281.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:06:13.185382 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271281.sst): second pass time = 1.39's, 77.910'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.552'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2905.734'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 308.207'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32321.500'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 77.910'MB/sec zip pipeline throughput = 77.910'MB/sec entries = 925568 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.01 avg-zval = 35.66 usrkeys = 925568 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.01 avg-zval = 35.66 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925568 bytes = 3470896 } seq expand size = 6478976 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0330 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.4434 value = 3.2816 dict = 1.00 all = 3.4124 } Zip/UnZip{ index = 0.0958 value = 0.3047 dict = 1.00 all = 0.2931 } ---------------------------- total value len = 10.555070 GB avg = 0.159 KB (by entry num) total key len = 1.835244 GB avg = 0.028 KB total ukey len = 1.279609 GB avg = 0.016 KB total ukey num = 0.079636971 Billion total entry num = 0.066444700 Billion write speed all = 0.420567053 MB/sec (with seq num) write speed all = 0.402524280 MB/sec (without seq num) 2024/11/23-17:06:13.233520 139957634799168 [compaction_job.cc:1948] [default] [JOB 31470] Generated table #271281: 925568 keys, 39332782 bytes, temperature: kWarm 2024/11/23-17:06:13.233591 139957634799168 EVENT_LOG_v1 {"time_micros": 1732352773233555, "cf_name": "default", "job": 31470, "event": "table_file_creation", "file_number": 271281, "file_size": 39332782, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776268, "table_properties": {"data_size": 33002608, "index_size": 1772544, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25915904, "raw_average_key_size": 28, "raw_value_size": 108301824, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925568, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352769, "oldest_key_time": 0, "file_creation_time": 1732352769, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271281, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:06:13.270048 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271282.sst) /dev/shm/ToplingTemp/Topling-126049-KND90s 2024/11/23-17:06:14.291966 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271282.sst): first pass time = 1.02's, 131.344'MB/sec 2024/11/23-17:06:15.584891 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271282.sst): old prealloc_size = 147639500, real_size = 32051360 2024/11/23-17:06:15.585500 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271282.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931242 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0300 GB avg-val = 116.13 avg-zval = 32.27 2024/11/23-17:06:15.586325 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271282.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:06:16.425318 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271282.sst): second pass time = 1.29's, 83.692'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.168'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3016.314'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 286.966'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33384.553'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.692'MB/sec zip pipeline throughput = 83.692'MB/sec entries = 931242 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.13 avg-zval = 32.27 usrkeys = 931242 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.13 avg-zval = 32.27 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931242 bytes = 3375760 } seq expand size = 6518694 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.5116 value = 3.5992 dict = 1.00 all = 3.6996 } Zip/UnZip{ index = 0.0951 value = 0.2778 dict = 1.00 all = 0.2703 } ---------------------------- total value len = 10.669731 GB avg = 0.158 KB (by entry num) total key len = 1.861319 GB avg = 0.028 KB total ukey len = 1.298234 GB avg = 0.016 KB total ukey num = 0.080568213 Billion total entry num = 0.067375942 Billion write speed all = 0.425297329 MB/sec (with seq num) write speed all = 0.407003693 MB/sec (without seq num) 2024/11/23-17:06:16.469717 139957634799168 [compaction_job.cc:1948] [default] [JOB 31470] Generated table #271282: 931242 keys, 36278632 bytes, temperature: kWarm 2024/11/23-17:06:16.469788 139957634799168 EVENT_LOG_v1 {"time_micros": 1732352776469753, "cf_name": "default", "job": 31470, "event": "table_file_creation", "file_number": 271282, "file_size": 36278632, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776300, "table_properties": {"data_size": 30046544, "index_size": 1771840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26074776, "raw_average_key_size": 28, "raw_value_size": 108142972, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931242, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352773, "oldest_key_time": 0, "file_creation_time": 1732352773, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271282, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:06:16.507233 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271283.sst) /dev/shm/ToplingTemp/Topling-126049-iDI9iq 2024/11/23-17:06:17.558435 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271283.sst): first pass time = 1.05's, 127.685'MB/sec 2024/11/23-17:06:18.889407 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271283.sst): old prealloc_size = 147639500, real_size = 35271872 2024/11/23-17:06:18.890025 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271283.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922697 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0333 GB avg-val = 117.46 avg-zval = 36.06 2024/11/23-17:06:18.892181 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271283.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:06:19.732832 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271283.sst): second pass time = 1.33's, 81.479'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.941'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2961.078'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.323'MB/sec write SST data time = 0.13's, 308.043'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34284.855'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.479'MB/sec zip pipeline throughput = 81.479'MB/sec entries = 922697 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.46 avg-zval = 36.06 usrkeys = 922697 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.46 avg-zval = 36.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922697 bytes = 3460128 } seq expand size = 6458879 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0333 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.4457 value = 3.2572 dict = 1.00 all = 3.3902 } Zip/UnZip{ index = 0.0957 value = 0.3070 dict = 1.00 all = 0.2950 } ---------------------------- total value len = 10.784572 GB avg = 0.158 KB (by entry num) total key len = 1.887154 GB avg = 0.028 KB total ukey len = 1.316688 GB avg = 0.016 KB total ukey num = 0.081490910 Billion total entry num = 0.068298639 Billion write speed all = 0.430023550 MB/sec (with seq num) write speed all = 0.411481468 MB/sec (without seq num) 2024/11/23-17:06:19.780782 139957634799168 [compaction_job.cc:1948] [default] [JOB 31470] Generated table #271283: 922697 keys, 39590206 bytes, temperature: kWarm 2024/11/23-17:06:19.780853 139957634799168 EVENT_LOG_v1 {"time_micros": 1732352779780815, "cf_name": "default", "job": 31470, "event": "table_file_creation", "file_number": 271283, "file_size": 39590206, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776316, "table_properties": {"data_size": 33274352, "index_size": 1766656, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25835516, "raw_average_key_size": 28, "raw_value_size": 108382282, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922697, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352776, "oldest_key_time": 0, "file_creation_time": 1732352776, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271283, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:06:19.825423 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271284.sst) /dev/shm/ToplingTemp/Topling-126049-wM3AGs 2024/11/23-17:06:20.714077 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271284.sst): first pass time = 0.89's, 151.043'MB/sec 2024/11/23-17:06:22.037387 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271284.sst): old prealloc_size = 147639500, real_size = 32875392 2024/11/23-17:06:22.037943 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271284.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931083 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 116.15 avg-zval = 33.13 2024/11/23-17:06:22.039113 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271284.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:06:22.892277 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271284.sst): second pass time = 1.32's, 81.773'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.537'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3367.138'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 295.925'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35696.524'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.773'MB/sec zip pipeline throughput = 81.773'MB/sec entries = 931083 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.15 avg-zval = 33.13 usrkeys = 931083 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.15 avg-zval = 33.13 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931083 bytes = 3491584 } seq expand size = 6517581 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.3760 value = 3.5059 dict = 1.00 all = 3.6062 } Zip/UnZip{ index = 0.0964 value = 0.2852 dict = 1.00 all = 0.2773 } ---------------------------- total value len = 10.899237 GB avg = 0.157 KB (by entry num) total key len = 1.913225 GB avg = 0.028 KB total ukey len = 1.335309 GB avg = 0.016 KB total ukey num = 0.082421993 Billion total entry num = 0.069229722 Billion write speed all = 0.434752887 MB/sec (with seq num) write speed all = 0.415960046 MB/sec (without seq num) 2024/11/23-17:06:22.937654 139957634799168 [compaction_job.cc:1948] [default] [JOB 31470] Generated table #271284: 931083 keys, 37218248 bytes, temperature: kWarm 2024/11/23-17:06:22.937723 139957634799168 EVENT_LOG_v1 {"time_micros": 1732352782937689, "cf_name": "default", "job": 31470, "event": "table_file_creation", "file_number": 271284, "file_size": 37218248, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012776344, "table_properties": {"data_size": 30847728, "index_size": 1794688, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26070324, "raw_average_key_size": 28, "raw_value_size": 108147522, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931083, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352779, "oldest_key_time": 0, "file_creation_time": 1732352779, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271284, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:06:22.974487 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271285.sst) /dev/shm/ToplingTemp/Topling-126049-Cxa5Zs 2024/11/23-17:06:23.937803 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271285.sst): first pass time = 0.96's, 139.334'MB/sec 2024/11/23-17:06:25.295854 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271285.sst): old prealloc_size = 147639500, real_size = 33985792 2024/11/23-17:06:25.296409 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271285.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927688 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0320 GB avg-val = 116.68 avg-zval = 34.47 2024/11/23-17:06:25.297035 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271285.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:06:26.118760 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271285.sst): second pass time = 1.36's, 79.749'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 45.100'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3350.806'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 290.473'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33804.330'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.749'MB/sec zip pipeline throughput = 79.749'MB/sec entries = 927688 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.68 avg-zval = 34.47 usrkeys = 927688 avg-key = 20.00 avg-zkey = 1.92 avg-val = 121.25 avg-zval = 34.47 TagRS{ kinds = RS_Key0_TagN bytes = 145040 } TagArray{ size = 606229 bytes = 2121824 } seq expand size = 4243603 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.4424 value = 3.3850 dict = 1.00 all = 3.6166 } Zip/UnZip{ index = 0.0958 value = 0.2954 dict = 1.00 all = 0.2765 } ---------------------------- total value len = 11.011724 GB avg = 0.157 KB (by entry num) total key len = 1.939200 GB avg = 0.028 KB total ukey len = 1.353863 GB avg = 0.016 KB total ukey num = 0.083349681 Billion total entry num = 0.070157410 Billion write speed all = 0.439403057 MB/sec (with seq num) write speed all = 0.420360474 MB/sec (without seq num) 2024/11/23-17:06:26.164629 139957634799168 [compaction_job.cc:1948] [default] [JOB 31470] Generated table #271285: 927688 keys, 37111766 bytes, temperature: kWarm 2024/11/23-17:06:26.164699 139957634799168 EVENT_LOG_v1 {"time_micros": 1732352786164663, "cf_name": "default", "job": 31470, "event": "table_file_creation", "file_number": 271285, "file_size": 37111766, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 31976944, "index_size": 1776768, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25975264, "raw_average_key_size": 28, "raw_value_size": 108242516, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927688, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352782, "oldest_key_time": 0, "file_creation_time": 1732352782, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271285, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:06:26.202698 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271286.sst) /dev/shm/ToplingTemp/Topling-126049-K9l5Wo 2024/11/23-17:06:27.202566 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271286.sst): first pass time = 1.00's, 134.241'MB/sec 2024/11/23-17:06:28.683262 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271286.sst): old prealloc_size = 147639500, real_size = 36793136 2024/11/23-17:06:28.683841 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271286.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926690 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0348 GB avg-val = 116.84 avg-zval = 37.55 2024/11/23-17:06:28.684096 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271286.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:06:29.399054 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271286.sst): second pass time = 1.48's, 73.161'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.72's, 53.479'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3152.051'MB/sec (index lex order gen) rebuild zvType time = 0.59's, 0.392'MB/sec write SST data time = 0.12's, 307.798'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32622.301'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.161'MB/sec zip pipeline throughput = 73.161'MB/sec entries = 926690 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.84 avg-zval = 37.55 usrkeys = 926690 avg-key = 20.00 avg-zkey = 1.90 avg-val = 118.00 avg-zval = 37.55 TagRS{ kinds = RS_Key0_TagN bytes = 144880 } TagArray{ size = 154398 bytes = 482512 } seq expand size = 1080786 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0348 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.5137 value = 3.1114 dict = 1.00 all = 3.5063 } Zip/UnZip{ index = 0.0951 value = 0.3214 dict = 1.00 all = 0.2852 } ---------------------------- total value len = 11.121075 GB avg = 0.156 KB (by entry num) total key len = 1.965147 GB avg = 0.028 KB total ukey len = 1.372397 GB avg = 0.016 KB total ukey num = 0.084276371 Billion total entry num = 0.071084100 Billion write speed all = 0.443944101 MB/sec (with seq num) write speed all = 0.424652136 MB/sec (without seq num) 2024/11/23-17:06:29.446316 139957634799168 [compaction_job.cc:1948] [default] [JOB 31470] Generated table #271286: 926690 keys, 38278769 bytes, temperature: kWarm 2024/11/23-17:06:29.446395 139957634799168 EVENT_LOG_v1 {"time_micros": 1732352789446352, "cf_name": "default", "job": 31470, "event": "table_file_creation", "file_number": 271286, "file_size": 38278769, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700812, "table_properties": {"data_size": 34798432, "index_size": 1762816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25947320, "raw_average_key_size": 28, "raw_value_size": 108270440, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926690, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352786, "oldest_key_time": 0, "file_creation_time": 1732352786, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271286, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:06:29.486317 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271287.sst) /dev/shm/ToplingTemp/Topling-126049-bhRfus 2024/11/23-17:06:30.530562 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271287.sst): first pass time = 1.04's, 128.537'MB/sec 2024/11/23-17:06:31.882577 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271287.sst): old prealloc_size = 147639500, real_size = 35557648 2024/11/23-17:06:31.883148 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271287.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925639 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0336 GB avg-val = 117.00 avg-zval = 36.27 2024/11/23-17:06:31.883778 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271287.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:06:32.686749 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271287.sst): second pass time = 1.35's, 80.153'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.80's, 48.478'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3194.548'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.342'MB/sec write SST data time = 0.13's, 307.302'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36421.482'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.153'MB/sec zip pipeline throughput = 80.153'MB/sec entries = 925639 avg-key = 28.00 avg-zkey = 1.89 avg-val = 117.00 avg-zval = 36.27 usrkeys = 925639 avg-key = 20.00 avg-zkey = 1.89 avg-val = 122.43 avg-zval = 36.27 TagRS{ kinds = RS_Key0_TagN bytes = 144720 } TagArray{ size = 717797 bytes = 2422576 } seq expand size = 5024579 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0336 GB dict = 1.08 MB all = 0.0390 GB } UnZip/Zip{ index = 10.5825 value = 3.2254 dict = 1.00 all = 3.4429 } Zip/UnZip{ index = 0.0945 value = 0.3100 dict = 1.00 all = 0.2904 } ---------------------------- total value len = 11.234399 GB avg = 0.156 KB (by entry num) total key len = 1.991065 GB avg = 0.028 KB total ukey len = 1.390910 GB avg = 0.016 KB total ukey num = 0.085202010 Billion total entry num = 0.072009739 Billion write speed all = 0.448617799 MB/sec (with seq num) write speed all = 0.429076799 MB/sec (without seq num) 2024/11/23-17:06:32.735783 139957634799168 [compaction_job.cc:1948] [default] [JOB 31470] Generated table #271287: 925639 keys, 38983454 bytes, temperature: kWarm 2024/11/23-17:06:32.735854 139957634799168 EVENT_LOG_v1 {"time_micros": 1732352792735817, "cf_name": "default", "job": 31470, "event": "table_file_creation", "file_number": 271287, "file_size": 38983454, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4133207002, "table_properties": {"data_size": 33576704, "index_size": 1749376, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25917892, "raw_average_key_size": 28, "raw_value_size": 108299918, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925639, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352789, "oldest_key_time": 0, "file_creation_time": 1732352789, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271287, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:06:32.774853 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271288.sst) /dev/shm/ToplingTemp/Topling-126049-dQg9oq 2024/11/23-17:06:33.671665 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271288.sst): first pass time = 0.90's, 149.666'MB/sec 2024/11/23-17:06:35.015890 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271288.sst): old prealloc_size = 147639500, real_size = 35511392 2024/11/23-17:06:35.016448 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271288.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928480 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0335 GB avg-val = 116.56 avg-zval = 36.09 2024/11/23-17:06:35.017196 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271288.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:06:35.842799 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271288.sst): second pass time = 1.34's, 80.553'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.627'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3241.801'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.13's, 313.662'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 28452.420'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.553'MB/sec zip pipeline throughput = 80.553'MB/sec entries = 928480 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.56 avg-zval = 36.09 usrkeys = 928480 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.56 avg-zval = 36.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928480 bytes = 3017568 } seq expand size = 6499360 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0335 GB dict = 1.08 MB all = 0.0394 GB } UnZip/Zip{ index = 10.5138 value = 3.2292 dict = 1.00 all = 3.4081 } Zip/UnZip{ index = 0.0951 value = 0.3097 dict = 1.00 all = 0.2934 } ---------------------------- total value len = 11.349119 GB avg = 0.156 KB (by entry num) total key len = 2.017063 GB avg = 0.028 KB total ukey len = 1.409479 GB avg = 0.016 KB total ukey num = 0.086130490 Billion total entry num = 0.072938219 Billion write speed all = 0.453342499 MB/sec (with seq num) write speed all = 0.433551660 MB/sec (without seq num) 2024/11/23-17:06:35.890566 139957634799168 [compaction_job.cc:1948] [default] [JOB 31470] Generated table #271288: 928480 keys, 39382356 bytes, temperature: kWarm 2024/11/23-17:06:35.890636 139957634799168 EVENT_LOG_v1 {"time_micros": 1732352795890600, "cf_name": "default", "job": 31470, "event": "table_file_creation", "file_number": 271288, "file_size": 39382356, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4072856654, "largest_seqno": 4133206912, "table_properties": {"data_size": 33512912, "index_size": 1766208, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25997440, "raw_average_key_size": 28, "raw_value_size": 108220368, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928480, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352792, "oldest_key_time": 0, "file_creation_time": 1732352792, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271288, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:06:35.927812 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271289.sst) /dev/shm/ToplingTemp/Topling-126049-a94VUq 2024/11/23-17:06:36.015481 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271289.sst): first pass time = 0.09's, 235.711'MB/sec 2024/11/23-17:06:36.211551 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271289.sst): old prealloc_size = 147639500, real_size = 5830736 2024/11/23-17:06:36.211691 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271289.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 143702 prefix = 4 raw-key = 0.0023 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0166 GB zip-val = 0.0055 GB avg-val = 115.74 avg-zval = 38.40 2024/11/23-17:06:36.211848 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271289.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.021 GB 2024/11/23-17:06:36.341935 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00/271289.sst): second pass time = 0.20's, 85.158'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.13's, 49.109'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2373.688'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.325'MB/sec write SST data time = 0.02's, 323.288'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.175'MB, throughput = 5497.250'MB/sec zip my value time = 0.20's, unzip length = 0.017'GB zip my value throughput = 85.158'MB/sec zip pipeline throughput = 85.158'MB/sec entries = 143702 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.74 avg-zval = 38.40 usrkeys = 143702 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.74 avg-zval = 38.40 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 143702 bytes = 431120 } seq expand size = 1005914 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0029 GB value = 0.0166 GB dict = 0.17 MB all = 0.0207 GB } __ZipSize{ index = 0.0003 GB value = 0.0055 GB dict = 0.17 MB all = 0.0064 GB } UnZip/Zip{ index = 10.3975 value = 3.0141 dict = 1.00 all = 3.2263 } Zip/UnZip{ index = 0.0962 value = 0.3318 dict = 1.00 all = 0.3100 } ---------------------------- total value len = 11.366757 GB avg = 0.156 KB (by entry num) total key len = 2.021086 GB avg = 0.028 KB total ukey len = 1.412353 GB avg = 0.016 KB total ukey num = 0.086274192 Billion total entry num = 0.073081921 Billion write speed all = 0.454069521 MB/sec (with seq num) write speed all = 0.434240025 MB/sec (without seq num) 2024/11/23-17:06:36.356084 139957634799168 [compaction_job.cc:1948] [default] [JOB 31470] Generated table #271289: 143702 keys, 6402463 bytes, temperature: kWarm 2024/11/23-17:06:36.356136 139957634799168 EVENT_LOG_v1 {"time_micros": 1732352796356108, "cf_name": "default", "job": 31470, "event": "table_file_creation", "file_number": 271289, "file_size": 6402463, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4119197640, "largest_seqno": 4133205375, "table_properties": {"data_size": 5518208, "index_size": 276416, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4023656, "raw_average_key_size": 28, "raw_value_size": 16632296, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 143702, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352795, "oldest_key_time": 0, "file_creation_time": 1732352795, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271289, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:06:36.368986 139957634799168 [compaction_job.cc:703] job-31470: subcompact[0], size: 0.844385 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31470/att-00 [271267,271268,271269,271270,271271,271272,271273,271274,271275,271276,271277,271278,271279,271280,271281,271282,271283,271284,271285,271286,271287,271288,271289] 2024/11/23-17:06:36.505053 139957634799168 (Original Log Time 2024/11/23-17:05:18.197208) EVENT_LOG_v1 {"time_micros": 1732352718197181, "job": 31470, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [271255], "files_L3": [271252, 271253, 271144, 271188, 271189, 271190, 271191, 271192, 271193, 271194, 271195, 271196, 271197, 271198, 271199, 271200, 271201, 271202, 271203, 271204, 271205, 271206, 271260], "score": 0.999972, "input_data_size": 867070523, "oldest_snapshot_seqno": 4133457899} 2024/11/23-17:06:36.505057 139957634799168 (Original Log Time 2024/11/23-17:06:36.503434) [compaction_job.cc:2022] [default] [JOB 31470] Compacted 1@2 + 23@3 files to L3 => 844384586 bytes 2024/11/23-17:06:36.505058 139957634799168 (Original Log Time 2024/11/23-17:06:36.504966) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 23 0 0 0] max score 0.03, MB/sec: 11.1 rd, 10.8 wr, level 3, files in(1, 23) out(23 +0 blob) MB in(27.1, 799.8 +0.0 blob) out(805.3 +0.0 blob), read-write-amplify(60.2) write-amplify(29.7) OK, records in: 20832475, records dropped: 108718 output_compression: Snappy 2024/11/23-17:06:36.505062 139957634799168 (Original Log Time 2024/11/23-17:06:36.505026) EVENT_LOG_v1 {"time_micros": 1732352796505005, "cf": "default", "job": 31470, "event": "compaction_finished", "compaction_time_micros": 78171784, "compaction_time_cpu_micros": 32363678, "output_level": 3, "num_output_files": 23, "total_output_size": 844384586, "num_input_records": 20832475, "num_output_records": 20723757, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 614667101, "file_range_sync_nanos": 0, "file_fsync_nanos": 18177, "file_prepare_write_nanos": 62787, "lsm_state": [0, 0, 0, 23, 0, 0, 0]} 2024/11/23-17:06:36.657378 139957634799168 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732352718,"executesMs":78442,"compactionJobId":31470,"attempt":0,"compactionInputRawBytes":2988885279,"compactionInputZipBytes":867070523,"compactionOutputRawBytes":0,"compactionOutputZipBytes":844384586,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-ca2428dbd7dc58c4200f645c94ef8879-615b8f38dece2fef-00"}}, response = 2024/11/23-17:06:36.657471 139957634799168 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-31470/att-00: olev 3, work 78.442 s, result 7.758 ms, install 1.644 ms, input{raw 2.784 GiB zip 826.903 MiB}