2024/11/23-01:24:24.764434 140024818968128 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00: fork to child time = 0.008936 sec 2024/11/23-01:24:24.765093 140024818968128 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00 2024/11/23-01:24:24.772086 140024818968128 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00 2024/11/23-01:24:24.773225 140024818968128 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-01:24:24.773699 140024818968128 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-01:24:24.821202 140024818968128 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-16314/att-00/MANIFEST-206347 2024/11/23-01:24:24.855471 140024818968128 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-16314/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 238550, last_sequence is 3704070949, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 238338 2024/11/23-01:24:24.855481 140024818968128 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-01:24:24.855608 140024818968128 [version_set.cc:5721] Creating manifest 238550 2024/11/23-01:24:24.886382 140024818968128 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-01:24:24.907959 140024818968128 [compaction_job.cc:2437] [default] [JOB 16314] Compacting 1@2 + 16@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-01:24:24.907977 140024818968128 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 12218 Base level 2, inputs: [238483(27MB)], [238536(35MB) 238537(6020KB) 238431(33MB) 238432(35MB) 238433(35MB) 238434(35MB) 238435(35MB) 238436(35MB) 238437(34MB) 238438(35MB) 238439(35MB) 238440(35MB) 238441(36MB) 238442(34MB) 238443(25MB) 238538(35MB)] 2024/11/23-01:24:24.915278 140024818968128 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-01:24:24.915311 140024818968128 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-01:24:24.915331 140024818968128 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-01:24:24.915336 140024818968128 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-01:24:24.915348 140024818968128 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-01:24:25.573719 140024818968128 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238551.sst) /dev/shm/ToplingTemp/Topling-366232-SxwufX 2024/11/23-01:24:26.122435 140024818968128 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238551.sst): first pass time = 0.55's, 244.628'MB/sec 2024/11/23-01:24:27.443940 140024818968128 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238551.sst): old prealloc_size = 147639500, real_size = 33116464 2024/11/23-01:24:27.444526 140024818968128 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238551.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928158 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.61 avg-zval = 33.53 2024/11/23-01:24:27.445346 140024818968128 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238551.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:24:28.395365 140024818968128 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238551.sst): second pass time = 1.32's, 81.949'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.241'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3143.903'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.283'MB/sec write SST data time = 0.13's, 287.193'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 29910.844'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.949'MB/sec zip pipeline throughput = 81.949'MB/sec entries = 928158 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.61 avg-zval = 33.53 usrkeys = 928158 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.61 avg-zval = 33.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928158 bytes = 3364592 } seq expand size = 6497106 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5106 value = 3.4780 dict = 1.00 all = 3.5950 } Zip/UnZip{ index = 0.0951 value = 0.2875 dict = 1.00 all = 0.2782 } ---------------------------- total value len = 0.114726 GB avg = 0.124 KB (by entry num) total key len = 0.025988 GB avg = 0.028 KB total ukey len = 0.018563 GB avg = 0.020 KB total ukey num = 0.000928158 Billion total entry num = 0.000928158 Billion write speed all = 0.002363951 MB/sec (with seq num) write speed all = 0.002239209 MB/sec (without seq num) 2024/11/23-01:24:28.428958 140024818968128 [compaction_job.cc:1948] [default] [JOB 16314] Generated table #238551: 928158 keys, 37334466 bytes, temperature: kWarm 2024/11/23-01:24:28.429051 140024818968128 EVENT_LOG_v1 {"time_micros": 1732296268428993, "cf_name": "default", "job": 16314, "event": "table_file_creation", "file_number": 238551, "file_size": 37334466, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701386, "largest_seqno": 3703685958, "table_properties": {"data_size": 31118112, "index_size": 1766144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25988424, "raw_average_key_size": 28, "raw_value_size": 108229348, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928158, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296265, "oldest_key_time": 0, "file_creation_time": 1732296265, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238551, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:24:28.470341 140024818968128 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238552.sst) /dev/shm/ToplingTemp/Topling-366232-PIiPZX 2024/11/23-01:24:29.312088 140024818968128 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238552.sst): first pass time = 0.84's, 159.460'MB/sec 2024/11/23-01:24:30.644832 140024818968128 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238552.sst): old prealloc_size = 147639500, real_size = 32439984 2024/11/23-01:24:30.645415 140024818968128 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238552.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928463 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0304 GB avg-val = 116.56 avg-zval = 32.77 2024/11/23-01:24:30.646182 140024818968128 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238552.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:24:31.568377 140024818968128 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238552.sst): second pass time = 1.33's, 81.298'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 39.443'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3179.835'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.292'MB/sec write SST data time = 0.13's, 283.934'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 29594.460'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.298'MB/sec zip pipeline throughput = 81.298'MB/sec entries = 928463 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.56 avg-zval = 32.77 usrkeys = 928463 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.56 avg-zval = 32.77 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928463 bytes = 3133584 } seq expand size = 6499241 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.4455 value = 3.5564 dict = 1.00 all = 3.6846 } Zip/UnZip{ index = 0.0957 value = 0.2812 dict = 1.00 all = 0.2714 } ---------------------------- total value len = 0.229447 GB avg = 0.124 KB (by entry num) total key len = 0.051985 GB avg = 0.028 KB total ukey len = 0.037132 GB avg = 0.020 KB total ukey num = 0.001856621 Billion total entry num = 0.001856621 Billion write speed all = 0.004727686 MB/sec (with seq num) write speed all = 0.004478176 MB/sec (without seq num) 2024/11/23-01:24:31.599642 140024818968128 [compaction_job.cc:1948] [default] [JOB 16314] Generated table #238552: 928463 keys, 36426442 bytes, temperature: kWarm 2024/11/23-01:24:31.599711 140024818968128 EVENT_LOG_v1 {"time_micros": 1732296271599677, "cf_name": "default", "job": 16314, "event": "table_file_creation", "file_number": 238552, "file_size": 36426442, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 3703685970, "table_properties": {"data_size": 30429984, "index_size": 1777728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25996964, "raw_average_key_size": 28, "raw_value_size": 108220850, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928463, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296268, "oldest_key_time": 0, "file_creation_time": 1732296268, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238552, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:24:31.638032 140024818968128 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238553.sst) /dev/shm/ToplingTemp/Topling-366232-dTgksY 2024/11/23-01:24:32.550546 140024818968128 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238553.sst): first pass time = 0.91's, 147.093'MB/sec 2024/11/23-01:24:33.880489 140024818968128 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238553.sst): old prealloc_size = 147639500, real_size = 33161344 2024/11/23-01:24:33.881067 140024818968128 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238553.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929832 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.35 avg-zval = 33.51 2024/11/23-01:24:33.881857 140024818968128 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238553.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:24:34.796176 140024818968128 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238553.sst): second pass time = 1.33's, 81.422'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.827'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3165.917'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.296'MB/sec write SST data time = 0.13's, 289.898'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 27364.750'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.422'MB/sec zip pipeline throughput = 81.422'MB/sec entries = 929832 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.35 avg-zval = 33.51 usrkeys = 929832 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.35 avg-zval = 33.51 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929832 bytes = 3370656 } seq expand size = 6508824 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5150 value = 3.4718 dict = 1.00 all = 3.5902 } Zip/UnZip{ index = 0.0951 value = 0.2880 dict = 1.00 all = 0.2785 } ---------------------------- total value len = 0.344138 GB avg = 0.124 KB (by entry num) total key len = 0.078021 GB avg = 0.028 KB total ukey len = 0.055729 GB avg = 0.020 KB total ukey num = 0.002786453 Billion total entry num = 0.002786453 Billion write speed all = 0.007091323 MB/sec (with seq num) write speed all = 0.006716874 MB/sec (without seq num) 2024/11/23-01:24:34.829480 140024818968128 [compaction_job.cc:1948] [default] [JOB 16314] Generated table #238553: 929832 keys, 37384270 bytes, temperature: kWarm 2024/11/23-01:24:34.829547 140024818968128 EVENT_LOG_v1 {"time_micros": 1732296274829513, "cf_name": "default", "job": 16314, "event": "table_file_creation", "file_number": 238553, "file_size": 37384270, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3703685984, "table_properties": {"data_size": 31160112, "index_size": 1768576, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26035296, "raw_average_key_size": 28, "raw_value_size": 108182476, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929832, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296271, "oldest_key_time": 0, "file_creation_time": 1732296271, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238553, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:24:34.866247 140024818968128 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238554.sst) /dev/shm/ToplingTemp/Topling-366232-PuRgJW 2024/11/23-01:24:35.658876 140024818968128 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238554.sst): first pass time = 0.79's, 169.342'MB/sec 2024/11/23-01:24:36.984348 140024818968128 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238554.sst): old prealloc_size = 147639500, real_size = 31386672 2024/11/23-01:24:36.984937 140024818968128 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238554.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926518 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0294 GB avg-val = 116.86 avg-zval = 31.72 2024/11/23-01:24:36.985719 140024818968128 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238554.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:24:37.849095 140024818968128 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238554.sst): second pass time = 1.32's, 81.734'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 41.035'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3138.314'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.13's, 275.774'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 30902.477'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.734'MB/sec zip pipeline throughput = 81.734'MB/sec entries = 926518 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.86 avg-zval = 31.72 usrkeys = 926518 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.86 avg-zval = 31.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926518 bytes = 3242832 } seq expand size = 6485626 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.5137 value = 3.6838 dict = 1.00 all = 3.7825 } Zip/UnZip{ index = 0.0951 value = 0.2715 dict = 1.00 all = 0.2644 } ---------------------------- total value len = 0.458899 GB avg = 0.124 KB (by entry num) total key len = 0.103963 GB avg = 0.028 KB total ukey len = 0.074259 GB avg = 0.020 KB total ukey num = 0.003712971 Billion total entry num = 0.003712971 Billion write speed all = 0.009454343 MB/sec (with seq num) write speed all = 0.008955411 MB/sec (without seq num) 2024/11/23-01:24:37.879189 140024818968128 [compaction_job.cc:1948] [default] [JOB 16314] Generated table #238554: 926518 keys, 35484264 bytes, temperature: kWarm 2024/11/23-01:24:37.879256 140024818968128 EVENT_LOG_v1 {"time_micros": 1732296277879222, "cf_name": "default", "job": 16314, "event": "table_file_creation", "file_number": 238554, "file_size": 35484264, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3563652023, "largest_seqno": 3703685998, "table_properties": {"data_size": 29392352, "index_size": 1762496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25942504, "raw_average_key_size": 28, "raw_value_size": 108275320, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926518, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296274, "oldest_key_time": 0, "file_creation_time": 1732296274, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238554, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:24:37.919435 140024818968128 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238555.sst) /dev/shm/ToplingTemp/Topling-366232-l6AgqV 2024/11/23-01:24:38.818283 140024818968128 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238555.sst): first pass time = 0.90's, 149.330'MB/sec 2024/11/23-01:24:40.116001 140024818968128 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238555.sst): old prealloc_size = 147639500, real_size = 31026128 2024/11/23-01:24:40.116593 140024818968128 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238555.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926536 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0290 GB avg-val = 116.86 avg-zval = 31.33 2024/11/23-01:24:40.117388 140024818968128 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238555.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:24:40.976224 140024818968128 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238555.sst): second pass time = 1.30's, 83.483'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 40.832'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3165.426'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 273.044'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 28814.058'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.483'MB/sec zip pipeline throughput = 83.483'MB/sec entries = 926536 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.86 avg-zval = 31.33 usrkeys = 926536 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.86 avg-zval = 31.33 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926536 bytes = 3242896 } seq expand size = 6485752 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0290 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.5135 value = 3.7295 dict = 1.00 all = 3.8213 } Zip/UnZip{ index = 0.0951 value = 0.2681 dict = 1.00 all = 0.2617 } ---------------------------- total value len = 0.573659 GB avg = 0.124 KB (by entry num) total key len = 0.129906 GB avg = 0.028 KB total ukey len = 0.092790 GB avg = 0.020 KB total ukey num = 0.004639507 Billion total entry num = 0.004639507 Billion write speed all = 0.011817107 MB/sec (with seq num) write speed all = 0.011193704 MB/sec (without seq num) 2024/11/23-01:24:41.007594 140024818968128 [compaction_job.cc:1948] [default] [JOB 16314] Generated table #238555: 926536 keys, 35123772 bytes, temperature: kWarm 2024/11/23-01:24:41.007663 140024818968128 EVENT_LOG_v1 {"time_micros": 1732296281007629, "cf_name": "default", "job": 16314, "event": "table_file_creation", "file_number": 238555, "file_size": 35123772, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3504428200, "largest_seqno": 3703686009, "table_properties": {"data_size": 29031744, "index_size": 1762560, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25943008, "raw_average_key_size": 28, "raw_value_size": 108274732, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926536, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296277, "oldest_key_time": 0, "file_creation_time": 1732296277, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238555, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:24:41.050310 140024818968128 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238556.sst) /dev/shm/ToplingTemp/Topling-366232-SjJg8W 2024/11/23-01:24:41.861700 140024818968128 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238556.sst): first pass time = 0.81's, 165.427'MB/sec 2024/11/23-01:24:43.154670 140024818968128 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238556.sst): old prealloc_size = 147639500, real_size = 31571728 2024/11/23-01:24:43.155263 140024818968128 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238556.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926439 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0296 GB avg-val = 116.87 avg-zval = 31.93 2024/11/23-01:24:43.156043 140024818968128 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238556.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:24:44.018653 140024818968128 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238556.sst): second pass time = 1.29's, 83.785'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 41.285'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3097.858'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.13's, 280.113'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 23930.449'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.785'MB/sec zip pipeline throughput = 83.785'MB/sec entries = 926439 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.87 avg-zval = 31.93 usrkeys = 926439 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.87 avg-zval = 31.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926439 bytes = 3242544 } seq expand size = 6485073 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5170 value = 3.6607 dict = 1.00 all = 3.7629 } Zip/UnZip{ index = 0.0951 value = 0.2732 dict = 1.00 all = 0.2658 } ---------------------------- total value len = 0.688422 GB avg = 0.124 KB (by entry num) total key len = 0.155846 GB avg = 0.028 KB total ukey len = 0.111319 GB avg = 0.020 KB total ukey num = 0.005565946 Billion total entry num = 0.005565946 Billion write speed all = 0.014179631 MB/sec (with seq num) write speed all = 0.013431783 MB/sec (without seq num) 2024/11/23-01:24:44.049979 140024818968128 [compaction_job.cc:1948] [default] [JOB 16314] Generated table #238556: 926439 keys, 35668820 bytes, temperature: kWarm 2024/11/23-01:24:44.050049 140024818968128 EVENT_LOG_v1 {"time_micros": 1732296284050014, "cf_name": "default", "job": 16314, "event": "table_file_creation", "file_number": 238556, "file_size": 35668820, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3508638461, "largest_seqno": 3703686020, "table_properties": {"data_size": 29578112, "index_size": 1761792, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25940292, "raw_average_key_size": 28, "raw_value_size": 108277502, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926439, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296281, "oldest_key_time": 0, "file_creation_time": 1732296281, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238556, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:24:44.098374 140024818968128 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238557.sst) /dev/shm/ToplingTemp/Topling-366232-WrH09V 2024/11/23-01:24:44.878116 140024818968128 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238557.sst): first pass time = 0.78's, 172.142'MB/sec 2024/11/23-01:24:46.221513 140024818968128 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238557.sst): old prealloc_size = 147639500, real_size = 32333536 2024/11/23-01:24:46.222105 140024818968128 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238557.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927958 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0303 GB avg-val = 116.64 avg-zval = 32.69 2024/11/23-01:24:46.222753 140024818968128 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238557.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:24:47.097637 140024818968128 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238557.sst): second pass time = 1.34's, 80.609'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 40.987'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3095.427'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.311'MB/sec write SST data time = 0.13's, 278.340'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32063.305'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.609'MB/sec zip pipeline throughput = 80.609'MB/sec entries = 927958 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.64 avg-zval = 32.69 usrkeys = 927958 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.39 avg-zval = 32.69 TagRS{ kinds = RS_Key0_TagN bytes = 145080 } TagArray{ size = 762507 bytes = 2573472 } seq expand size = 5337549 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0359 GB } UnZip/Zip{ index = 10.5148 value = 3.5678 dict = 1.00 all = 3.7377 } Zip/UnZip{ index = 0.0951 value = 0.2803 dict = 1.00 all = 0.2675 } ---------------------------- total value len = 0.801994 GB avg = 0.123 KB (by entry num) total key len = 0.181829 GB avg = 0.028 KB total ukey len = 0.129878 GB avg = 0.020 KB total ukey num = 0.006493904 Billion total entry num = 0.006493904 Billion write speed all = 0.016522634 MB/sec (with seq num) write speed all = 0.015650149 MB/sec (without seq num) 2024/11/23-01:24:47.128644 140024818968128 [compaction_job.cc:1948] [default] [JOB 16314] Generated table #238557: 927958 keys, 35909604 bytes, temperature: kWarm 2024/11/23-01:24:47.128711 140024818968128 EVENT_LOG_v1 {"time_micros": 1732296287128678, "cf_name": "default", "job": 16314, "event": "table_file_creation", "file_number": 238557, "file_size": 35909604, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3703686137, "table_properties": {"data_size": 30336336, "index_size": 1765056, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25982824, "raw_average_key_size": 28, "raw_value_size": 108235040, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927958, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296284, "oldest_key_time": 0, "file_creation_time": 1732296284, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238557, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:24:47.164359 140024818968128 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238558.sst) /dev/shm/ToplingTemp/Topling-366232-UJuDwY 2024/11/23-01:24:47.961439 140024818968128 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238558.sst): first pass time = 0.80's, 168.398'MB/sec 2024/11/23-01:24:49.363164 140024818968128 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238558.sst): old prealloc_size = 147639500, real_size = 34978640 2024/11/23-01:24:49.363771 140024818968128 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238558.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1081491 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1039 GB zip-val = 0.0327 GB avg-val = 96.10 avg-zval = 30.19 2024/11/23-01:24:49.363827 140024818968128 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238558.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:24:49.986548 140024818968128 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238558.sst): second pass time = 1.40's, 74.216'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.62's, 57.358'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3435.371'MB/sec (index lex order gen) rebuild zvType time = 0.48's, 0.558'MB/sec write SST data time = 0.14's, 258.745'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.036'MB, throughput = 28528.497'MB/sec zip my value time = 1.40's, unzip length = 0.104'GB zip my value throughput = 74.216'MB/sec zip pipeline throughput = 74.216'MB/sec entries = 1081491 avg-key = 28.00 avg-zkey = 1.90 avg-val = 96.10 avg-zval = 30.19 usrkeys = 1081491 avg-key = 20.00 avg-zkey = 1.90 avg-val = 96.10 avg-zval = 30.19 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0216 GB value = 0.1039 GB dict = 1.04 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0327 GB dict = 1.04 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5161 value = 3.1832 dict = 1.00 all = 3.7538 } Zip/UnZip{ index = 0.0951 value = 0.3141 dict = 1.00 all = 0.2664 } ---------------------------- total value len = 0.905931 GB avg = 0.120 KB (by entry num) total key len = 0.212111 GB avg = 0.028 KB total ukey len = 0.151508 GB avg = 0.020 KB total ukey num = 0.007575395 Billion total entry num = 0.007575395 Billion write speed all = 0.018775819 MB/sec (with seq num) write speed all = 0.017758080 MB/sec (without seq num) 2024/11/23-01:24:50.021219 140024818968128 [compaction_job.cc:1948] [default] [JOB 16314] Generated table #238558: 1081491 keys, 35755325 bytes, temperature: kWarm 2024/11/23-01:24:50.021289 140024818968128 EVENT_LOG_v1 {"time_micros": 1732296290021253, "cf_name": "default", "job": 16314, "event": "table_file_creation", "file_number": 238558, "file_size": 35755325, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 32651264, "index_size": 2056832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30281748, "raw_average_key_size": 28, "raw_value_size": 103936115, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1081491, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296287, "oldest_key_time": 0, "file_creation_time": 1732296287, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238558, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:24:50.058766 140024818968128 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238559.sst) /dev/shm/ToplingTemp/Topling-366232-gpFuQY 2024/11/23-01:24:50.951776 140024818968128 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238559.sst): first pass time = 0.89's, 150.307'MB/sec 2024/11/23-01:24:52.278699 140024818968128 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238559.sst): old prealloc_size = 147639500, real_size = 32883408 2024/11/23-01:24:52.279277 140024818968128 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238559.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927757 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.67 avg-zval = 33.29 2024/11/23-01:24:52.280025 140024818968128 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238559.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:24:53.161662 140024818968128 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238559.sst): second pass time = 1.33's, 81.618'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 41.607'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3179.424'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.308'MB/sec write SST data time = 0.13's, 282.589'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31565.466'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.618'MB/sec zip pipeline throughput = 81.618'MB/sec entries = 927757 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.67 avg-zval = 33.29 usrkeys = 927757 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.60 avg-zval = 33.29 TagRS{ kinds = RS_Key0_TagN bytes = 145080 } TagArray{ size = 786296 bytes = 2850336 } seq expand size = 5504072 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5114 value = 3.5045 dict = 1.00 all = 3.6535 } Zip/UnZip{ index = 0.0951 value = 0.2853 dict = 1.00 all = 0.2737 } ---------------------------- total value len = 1.019675 GB avg = 0.120 KB (by entry num) total key len = 0.238088 GB avg = 0.028 KB total ukey len = 0.170063 GB avg = 0.020 KB total ukey num = 0.008503152 Billion total entry num = 0.008503152 Billion write speed all = 0.021121110 MB/sec (with seq num) write speed all = 0.019978790 MB/sec (without seq num) 2024/11/23-01:24:53.193019 140024818968128 [compaction_job.cc:1948] [default] [JOB 16314] Generated table #238559: 927757 keys, 36736444 bytes, temperature: kWarm 2024/11/23-01:24:53.193084 140024818968128 EVENT_LOG_v1 {"time_micros": 1732296293193052, "cf_name": "default", "job": 16314, "event": "table_file_creation", "file_number": 238559, "file_size": 36736444, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3703686162, "table_properties": {"data_size": 30886016, "index_size": 1765248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25977196, "raw_average_key_size": 28, "raw_value_size": 108240578, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927757, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296290, "oldest_key_time": 0, "file_creation_time": 1732296290, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238559, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:24:53.230333 140024818968128 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238560.sst) /dev/shm/ToplingTemp/Topling-366232-JNbGTU 2024/11/23-01:24:54.089853 140024818968128 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238560.sst): first pass time = 0.86's, 156.162'MB/sec 2024/11/23-01:24:55.420254 140024818968128 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238560.sst): old prealloc_size = 147639500, real_size = 32820400 2024/11/23-01:24:55.420836 140024818968128 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238560.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926540 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0308 GB avg-val = 116.86 avg-zval = 33.27 2024/11/23-01:24:55.421595 140024818968128 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238560.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:24:56.332291 140024818968128 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238560.sst): second pass time = 1.33's, 81.460'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 40.607'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3154.196'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.295'MB/sec write SST data time = 0.13's, 290.974'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33442.931'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.460'MB/sec zip pipeline throughput = 81.460'MB/sec entries = 926540 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.86 avg-zval = 33.27 usrkeys = 926540 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.86 avg-zval = 33.27 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926540 bytes = 3358720 } seq expand size = 6485780 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5128 value = 3.5125 dict = 1.00 all = 3.6242 } Zip/UnZip{ index = 0.0951 value = 0.2847 dict = 1.00 all = 0.2759 } ---------------------------- total value len = 1.134436 GB avg = 0.120 KB (by entry num) total key len = 0.264031 GB avg = 0.028 KB total ukey len = 0.188594 GB avg = 0.020 KB total ukey num = 0.009429692 Billion total entry num = 0.009429692 Billion write speed all = 0.023482637 MB/sec (with seq num) write speed all = 0.022215913 MB/sec (without seq num) 2024/11/23-01:24:56.365196 140024818968128 [compaction_job.cc:1948] [default] [JOB 16314] Generated table #238560: 926540 keys, 37033968 bytes, temperature: kWarm 2024/11/23-01:24:56.365261 140024818968128 EVENT_LOG_v1 {"time_micros": 1732296296365229, "cf_name": "default", "job": 16314, "event": "table_file_creation", "file_number": 238560, "file_size": 37033968, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3375334738, "largest_seqno": 3703686172, "table_properties": {"data_size": 30825888, "index_size": 1762688, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25943120, "raw_average_key_size": 28, "raw_value_size": 108274692, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926540, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296293, "oldest_key_time": 0, "file_creation_time": 1732296293, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238560, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:24:56.404449 140024818968128 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238561.sst) /dev/shm/ToplingTemp/Topling-366232-r7kFgV 2024/11/23-01:24:57.311983 140024818968128 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238561.sst): first pass time = 0.91's, 147.900'MB/sec 2024/11/23-01:24:58.619605 140024818968128 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238561.sst): old prealloc_size = 147639500, real_size = 30802592 2024/11/23-01:24:58.620195 140024818968128 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238561.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926558 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0288 GB avg-val = 116.86 avg-zval = 31.09 2024/11/23-01:24:58.620971 140024818968128 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238561.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:24:59.469373 140024818968128 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238561.sst): second pass time = 1.31's, 82.877'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 41.208'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3101.027'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 276.061'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 30020.725'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.877'MB/sec zip pipeline throughput = 82.877'MB/sec entries = 926558 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.86 avg-zval = 31.09 usrkeys = 926558 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.86 avg-zval = 31.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926558 bytes = 3358784 } seq expand size = 6485906 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0288 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.5134 value = 3.7585 dict = 1.00 all = 3.8330 } Zip/UnZip{ index = 0.0951 value = 0.2661 dict = 1.00 all = 0.2609 } ---------------------------- total value len = 1.249196 GB avg = 0.121 KB (by entry num) total key len = 0.289975 GB avg = 0.028 KB total ukey len = 0.207125 GB avg = 0.020 KB total ukey num = 0.010356250 Billion total entry num = 0.010356250 Billion write speed all = 0.025843930 MB/sec (with seq num) write speed all = 0.024452811 MB/sec (without seq num) 2024/11/23-01:24:59.500799 140024818968128 [compaction_job.cc:1948] [default] [JOB 16314] Generated table #238561: 926558 keys, 35016164 bytes, temperature: kWarm 2024/11/23-01:24:59.500865 140024818968128 EVENT_LOG_v1 {"time_micros": 1732296299500831, "cf_name": "default", "job": 16314, "event": "table_file_creation", "file_number": 238561, "file_size": 35016164, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3703686185, "table_properties": {"data_size": 28808144, "index_size": 1762624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25943624, "raw_average_key_size": 28, "raw_value_size": 108274140, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926558, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296296, "oldest_key_time": 0, "file_creation_time": 1732296296, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238561, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:24:59.541481 140024818968128 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238562.sst) /dev/shm/ToplingTemp/Topling-366232-92wSzX 2024/11/23-01:25:00.452282 140024818968128 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238562.sst): first pass time = 0.91's, 147.370'MB/sec 2024/11/23-01:25:01.801236 140024818968128 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238562.sst): old prealloc_size = 147639500, real_size = 34378336 2024/11/23-01:25:01.801839 140024818968128 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238562.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926849 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0324 GB avg-val = 116.81 avg-zval = 34.93 2024/11/23-01:25:01.802591 140024818968128 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238562.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:25:02.714764 140024818968128 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238562.sst): second pass time = 1.35's, 80.299'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 42.247'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3083.345'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.295'MB/sec write SST data time = 0.13's, 303.320'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31067.403'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.299'MB/sec zip pipeline throughput = 80.299'MB/sec entries = 926849 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.81 avg-zval = 34.93 usrkeys = 926849 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.81 avg-zval = 34.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926849 bytes = 3359840 } seq expand size = 6487943 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0324 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.4424 value = 3.3445 dict = 1.00 all = 3.4778 } Zip/UnZip{ index = 0.0958 value = 0.2990 dict = 1.00 all = 0.2875 } ---------------------------- total value len = 1.363950 GB avg = 0.121 KB (by entry num) total key len = 0.315927 GB avg = 0.028 KB total ukey len = 0.225662 GB avg = 0.020 KB total ukey num = 0.011283099 Billion total entry num = 0.011283099 Billion write speed all = 0.028204958 MB/sec (with seq num) write speed all = 0.026689421 MB/sec (without seq num) 2024/11/23-01:25:02.750446 140024818968128 [compaction_job.cc:1948] [default] [JOB 16314] Generated table #238562: 926849 keys, 38592672 bytes, temperature: kWarm 2024/11/23-01:25:02.750513 140024818968128 EVENT_LOG_v1 {"time_micros": 1732296302750479, "cf_name": "default", "job": 16314, "event": "table_file_creation", "file_number": 238562, "file_size": 38592672, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3275013549, "largest_seqno": 3703686216, "table_properties": {"data_size": 32371280, "index_size": 1775168, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25951772, "raw_average_key_size": 28, "raw_value_size": 108266078, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926849, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296299, "oldest_key_time": 0, "file_creation_time": 1732296299, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238562, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:25:02.786618 140024818968128 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238563.sst) /dev/shm/ToplingTemp/Topling-366232-ap5CyX 2024/11/23-01:25:03.585552 140024818968128 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238563.sst): first pass time = 0.80's, 168.006'MB/sec 2024/11/23-01:25:04.924090 140024818968128 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238563.sst): old prealloc_size = 147639500, real_size = 33636064 2024/11/23-01:25:04.924608 140024818968128 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238563.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921942 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0316 GB avg-val = 117.58 avg-zval = 34.32 2024/11/23-01:25:04.925395 140024818968128 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238563.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:25:05.852440 140024818968128 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238563.sst): second pass time = 1.34's, 81.037'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.758'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3560.338'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.288'MB/sec write SST data time = 0.13's, 296.272'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 30261.788'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.037'MB/sec zip pipeline throughput = 81.037'MB/sec entries = 921942 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.58 avg-zval = 34.32 usrkeys = 921942 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.58 avg-zval = 34.32 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921942 bytes = 3342048 } seq expand size = 6453594 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.4447 value = 3.4261 dict = 1.00 all = 3.5473 } Zip/UnZip{ index = 0.0957 value = 0.2919 dict = 1.00 all = 0.2819 } ---------------------------- total value len = 1.478807 GB avg = 0.121 KB (by entry num) total key len = 0.341741 GB avg = 0.028 KB total ukey len = 0.244101 GB avg = 0.020 KB total ukey num = 0.012205041 Billion total entry num = 0.012205041 Billion write speed all = 0.030565205 MB/sec (with seq num) write speed all = 0.028925920 MB/sec (without seq num) 2024/11/23-01:25:05.887255 140024818968128 [compaction_job.cc:1948] [default] [JOB 16314] Generated table #238563: 921942 keys, 37836860 bytes, temperature: kWarm 2024/11/23-01:25:05.887322 140024818968128 EVENT_LOG_v1 {"time_micros": 1732296305887289, "cf_name": "default", "job": 16314, "event": "table_file_creation", "file_number": 238563, "file_size": 37836860, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3339428666, "largest_seqno": 3703686227, "table_properties": {"data_size": 31640016, "index_size": 1765376, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25814376, "raw_average_key_size": 28, "raw_value_size": 108403380, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921942, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296302, "oldest_key_time": 0, "file_creation_time": 1732296302, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238563, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:25:05.925008 140024818968128 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238564.sst) /dev/shm/ToplingTemp/Topling-366232-s8paMW 2024/11/23-01:25:06.978864 140024818968128 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238564.sst): first pass time = 1.05's, 127.364'MB/sec 2024/11/23-01:25:08.319151 140024818968128 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238564.sst): old prealloc_size = 147639500, real_size = 33385920 2024/11/23-01:25:08.319742 140024818968128 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238564.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926434 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.88 avg-zval = 33.87 2024/11/23-01:25:08.320462 140024818968128 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238564.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:25:09.267891 140024818968128 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238564.sst): second pass time = 1.34's, 80.828'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.387'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3125.322'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.282'MB/sec write SST data time = 0.13's, 293.251'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33442.728'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.828'MB/sec zip pipeline throughput = 80.828'MB/sec entries = 926434 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.88 avg-zval = 33.87 usrkeys = 926434 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.88 avg-zval = 33.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926434 bytes = 3126736 } seq expand size = 6485038 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4434 value = 3.4505 dict = 1.00 all = 3.5918 } Zip/UnZip{ index = 0.0958 value = 0.2898 dict = 1.00 all = 0.2784 } ---------------------------- total value len = 1.593569 GB avg = 0.121 KB (by entry num) total key len = 0.367681 GB avg = 0.028 KB total ukey len = 0.262630 GB avg = 0.020 KB total ukey num = 0.013131475 Billion total entry num = 0.013131475 Billion write speed all = 0.032925579 MB/sec (with seq num) write speed all = 0.031161963 MB/sec (without seq num) 2024/11/23-01:25:09.300319 140024818968128 [compaction_job.cc:1948] [default] [JOB 16314] Generated table #238564: 926434 keys, 37367496 bytes, temperature: kWarm 2024/11/23-01:25:09.300385 140024818968128 EVENT_LOG_v1 {"time_micros": 1732296309300351, "cf_name": "default", "job": 16314, "event": "table_file_creation", "file_number": 238564, "file_size": 37367496, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3610601794, "largest_seqno": 3703686240, "table_properties": {"data_size": 31379888, "index_size": 1774208, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25940152, "raw_average_key_size": 28, "raw_value_size": 108277616, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926434, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296305, "oldest_key_time": 0, "file_creation_time": 1732296305, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238564, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:25:09.337981 140024818968128 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238565.sst) /dev/shm/ToplingTemp/Topling-366232-Jk96HV 2024/11/23-01:25:09.874842 140024818968128 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238565.sst): first pass time = 0.54's, 246.503'MB/sec 2024/11/23-01:25:11.221522 140024818968128 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238565.sst): old prealloc_size = 147639500, real_size = 32811664 2024/11/23-01:25:11.222097 140024818968128 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238565.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 913677 prefix = 4 raw-key = 0.0146 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1067 GB zip-val = 0.0308 GB avg-val = 116.83 avg-zval = 33.76 2024/11/23-01:25:11.222843 140024818968128 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238565.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.132 GB 2024/11/23-01:25:12.164715 140024818968128 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00/238565.sst): second pass time = 1.35's, 79.333'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.076'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3152.610'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.280'MB/sec write SST data time = 0.13's, 290.237'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.068'MB, throughput = 31250.527'MB/sec zip my value time = 1.35's, unzip length = 0.107'GB zip my value throughput = 79.333'MB/sec zip pipeline throughput = 79.333'MB/sec entries = 913677 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.83 avg-zval = 33.76 usrkeys = 913677 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.83 avg-zval = 33.76 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 913677 bytes = 3197888 } seq expand size = 6395739 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0183 GB value = 0.1067 GB dict = 1.07 MB all = 0.1323 GB } __ZipSize{ index = 0.0017 GB value = 0.0308 GB dict = 1.07 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5158 value = 3.4606 dict = 1.00 all = 3.5905 } Zip/UnZip{ index = 0.0951 value = 0.2890 dict = 1.00 all = 0.2785 } ---------------------------- total value len = 1.706709 GB avg = 0.122 KB (by entry num) total key len = 0.393264 GB avg = 0.028 KB total ukey len = 0.280903 GB avg = 0.020 KB total ukey num = 0.014045152 Billion total entry num = 0.014045152 Billion write speed all = 0.035252751 MB/sec (with seq num) write speed all = 0.033366517 MB/sec (without seq num) 2024/11/23-01:25:12.200535 140024818968128 [compaction_job.cc:1948] [default] [JOB 16314] Generated table #238565: 913677 keys, 36855164 bytes, temperature: kWarm 2024/11/23-01:25:12.200610 140024818968128 EVENT_LOG_v1 {"time_micros": 1732296312200568, "cf_name": "default", "job": 16314, "event": "table_file_creation", "file_number": 238565, "file_size": 36855164, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 3703686258, "table_properties": {"data_size": 30845312, "index_size": 1737728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25582956, "raw_average_key_size": 28, "raw_value_size": 106744170, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 913677, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296309, "oldest_key_time": 0, "file_creation_time": 1732296309, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238565, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:25:12.229040 140024818968128 [compaction_job.cc:703] job-16314: subcompact[0], size: 0.548526 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16314/att-00 [238551,238552,238553,238554,238555,238556,238557,238558,238559,238560,238561,238562,238563,238564,238565] 2024/11/23-01:25:12.324118 140024818968128 (Original Log Time 2024/11/23-01:24:24.908033) EVENT_LOG_v1 {"time_micros": 1732296264907992, "job": 16314, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [238483], "files_L3": [238536, 238537, 238431, 238432, 238433, 238434, 238435, 238436, 238437, 238438, 238439, 238440, 238441, 238442, 238443, 238538], "score": 0.997547, "input_data_size": 579645096, "oldest_snapshot_seqno": 3704070885} 2024/11/23-01:25:12.324122 140024818968128 (Original Log Time 2024/11/23-01:25:12.323259) [compaction_job.cc:2022] [default] [JOB 16314] Compacted 1@2 + 16@3 files to L3 => 548525731 bytes 2024/11/23-01:25:12.324123 140024818968128 (Original Log Time 2024/11/23-01:25:12.324049) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 12.2 rd, 11.6 wr, level 3, files in(1, 16) out(15 +0 blob) MB in(27.2, 525.6 +0.0 blob) out(523.1 +0.0 blob), read-write-amplify(39.6) write-amplify(19.2) OK, records in: 14134371, records dropped: 89219 output_compression: Snappy 2024/11/23-01:25:12.324130 140024818968128 (Original Log Time 2024/11/23-01:25:12.324095) EVENT_LOG_v1 {"time_micros": 1732296312324073, "cf": "default", "job": 16314, "event": "compaction_finished", "compaction_time_micros": 47321019, "compaction_time_cpu_micros": 21514090, "output_level": 3, "num_output_files": 15, "total_output_size": 548525731, "num_input_records": 14134371, "num_output_records": 14045152, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 341201943, "file_range_sync_nanos": 0, "file_fsync_nanos": 14962, "file_prepare_write_nanos": 37298, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/23-01:25:12.463061 140024818968128 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732296264,"executesMs":47553,"compactionJobId":16314,"attempt":0,"compactionInputRawBytes":2024045391,"compactionInputZipBytes":579645096,"compactionOutputRawBytes":0,"compactionOutputZipBytes":548525731,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-48311e4c1f919139797c7be8376a5c4b-6d2f2a3c855ce22b-00"}}, response = 2024/11/23-01:25:12.463145 140024818968128 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-16314/att-00: olev 3, work 47.554 s, result 4.923 ms, install 0.892 ms, input{raw 1.885 GiB zip 552.793 MiB}