2024/11/23-18:49:20.190407 139957542479424 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00: fork to child time = 0.007944 sec 2024/11/23-18:49:20.191216 139957542479424 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00 2024/11/23-18:49:20.198811 139957542479424 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00 2024/11/23-18:49:20.199736 139957542479424 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-18:49:20.200016 139957542479424 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-18:49:20.254275 139957542479424 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-33516/att-00/MANIFEST-206347 2024/11/23-18:49:20.301671 139957542479424 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-33516/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 275725, last_sequence is 4195341036, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 275583 2024/11/23-18:49:20.301689 139957542479424 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-18:49:20.301882 139957542479424 [version_set.cc:5721] Creating manifest 275725 2024/11/23-18:49:20.336709 139957542479424 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-18:49:20.361413 139957542479424 [compaction_job.cc:2437] [default] [JOB 33516] Compacting 1@2 + 19@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-18:49:20.361444 139957542479424 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 25097 Base level 2, inputs: [275699(27MB)], [275640(37MB) 275641(38MB) 275642(36MB) 275643(38MB) 275644(35MB) 275645(36MB) 275646(34MB) 275647(35MB) 275648(37MB) 275649(34MB) 275650(36MB) 275651(35MB) 275652(35MB) 275653(36MB) 275654(37MB) 275655(37MB) 275656(36MB) 275662(35MB) 275720(35MB)] 2024/11/23-18:49:20.370030 139957542479424 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-18:49:20.370073 139957542479424 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-18:49:20.370101 139957542479424 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-18:49:20.370110 139957542479424 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-18:49:20.370128 139957542479424 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-18:49:21.092412 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275726.sst) /dev/shm/ToplingTemp/Topling-155990-6X0EHs 2024/11/23-18:49:21.673080 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275726.sst): first pass time = 0.58's, 231.161'MB/sec 2024/11/23-18:49:23.309522 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275726.sst): old prealloc_size = 147639500, real_size = 34529904 2024/11/23-18:49:23.310104 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275726.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927775 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0325 GB avg-val = 116.67 avg-zval = 35.07 2024/11/23-18:49:23.310920 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275726.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:49:24.379227 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275726.sst): second pass time = 1.64's, 66.179'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.07's, 36.118'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3153.836'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.265'MB/sec write SST data time = 0.20's, 197.751'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36035.624'MB/sec zip my value time = 1.64's, unzip length = 0.108'GB zip my value throughput = 66.179'MB/sec zip pipeline throughput = 96.953'MB/sec entries = 927775 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.67 avg-zval = 35.07 usrkeys = 927775 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.67 avg-zval = 35.07 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927775 bytes = 3247232 } seq expand size = 6494425 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.5123 value = 3.3271 dict = 1.00 all = 3.4740 } Zip/UnZip{ index = 0.0951 value = 0.3006 dict = 1.00 all = 0.2879 } ---------------------------- total value len = 8.963354 GB avg = 0.168 KB (by entry num) total key len = 1.466799 GB avg = 0.028 KB total ukey len = 1.016434 GB avg = 0.015 KB total ukey num = 0.066478210 Billion total entry num = 0.053285939 Billion write speed all = 0.292553150 MB/sec (with seq num) write speed all = 0.280596302 MB/sec (without seq num) 2024/11/23-18:49:24.426763 139957542479424 [compaction_job.cc:1948] [default] [JOB 33516] Generated table #275726: 927775 keys, 38635116 bytes, temperature: kWarm 2024/11/23-18:49:24.426843 139957542479424 EVENT_LOG_v1 {"time_micros": 1732358964426791, "cf_name": "default", "job": 33516, "event": "table_file_creation", "file_number": 275726, "file_size": 38635116, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3967285072, "largest_seqno": 4195146920, "table_properties": {"data_size": 32532640, "index_size": 1765120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25977700, "raw_average_key_size": 28, "raw_value_size": 108240054, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927775, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358961, "oldest_key_time": 0, "file_creation_time": 1732358961, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275726, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:49:24.463324 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275727.sst) /dev/shm/ToplingTemp/Topling-155990-y9vOmp 2024/11/23-18:49:25.419218 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275727.sst): first pass time = 0.96's, 140.415'MB/sec 2024/11/23-18:49:27.030679 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275727.sst): old prealloc_size = 147639500, real_size = 36546304 2024/11/23-18:49:27.031305 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275727.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932846 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0345 GB avg-val = 115.88 avg-zval = 37.00 2024/11/23-18:49:27.031929 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275727.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:49:27.993237 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275727.sst): second pass time = 1.61's, 67.114'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 41.389'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2977.597'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.281'MB/sec write SST data time = 0.13's, 302.921'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31077.291'MB/sec zip my value time = 1.61's, unzip length = 0.108'GB zip my value throughput = 67.114'MB/sec zip pipeline throughput = 67.114'MB/sec entries = 932846 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.88 avg-zval = 37.00 usrkeys = 932846 avg-key = 20.00 avg-zkey = 1.93 avg-val = 120.62 avg-zval = 37.00 TagRS{ kinds = RS_Key0_TagN bytes = 145840 } TagArray{ size = 632196 bytes = 2291728 } seq expand size = 4425372 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0398 GB } UnZip/Zip{ index = 10.3771 value = 3.1319 dict = 1.00 all = 3.3691 } Zip/UnZip{ index = 0.0964 value = 0.3193 dict = 1.00 all = 0.2968 } ---------------------------- total value len = 9.075878 GB avg = 0.167 KB (by entry num) total key len = 1.492918 GB avg = 0.028 KB total ukey len = 1.035090 GB avg = 0.015 KB total ukey num = 0.067411056 Billion total entry num = 0.054218785 Billion write speed all = 0.296411876 MB/sec (with seq num) write speed all = 0.284246939 MB/sec (without seq num) 2024/11/23-18:49:28.043093 139957542479424 [compaction_job.cc:1948] [default] [JOB 33516] Generated table #275727: 932846 keys, 39838186 bytes, temperature: kWarm 2024/11/23-18:49:28.043160 139957542479424 EVENT_LOG_v1 {"time_micros": 1732358968043126, "cf_name": "default", "job": 33516, "event": "table_file_creation", "file_number": 275727, "file_size": 39838186, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776165, "table_properties": {"data_size": 34514992, "index_size": 1797888, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26119688, "raw_average_key_size": 28, "raw_value_size": 108098100, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932846, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358964, "oldest_key_time": 0, "file_creation_time": 1732358964, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275727, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:49:28.082224 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275728.sst) /dev/shm/ToplingTemp/Topling-155990-0Aq4xq 2024/11/23-18:49:29.006108 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275728.sst): first pass time = 0.92's, 145.283'MB/sec 2024/11/23-18:49:30.650864 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275728.sst): old prealloc_size = 147639500, real_size = 34337808 2024/11/23-18:49:30.651413 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275728.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925521 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0323 GB avg-val = 117.02 avg-zval = 34.94 2024/11/23-18:49:30.652440 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275728.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:49:31.677960 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275728.sst): second pass time = 1.64's, 65.887'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 37.766'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3369.183'MB/sec (index lex order gen) rebuild zvType time = 0.90's, 0.258'MB/sec write SST data time = 0.13's, 297.931'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34955.545'MB/sec zip my value time = 1.64's, unzip length = 0.108'GB zip my value throughput = 65.887'MB/sec zip pipeline throughput = 65.887'MB/sec entries = 925521 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.02 avg-zval = 34.94 usrkeys = 925521 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.97 avg-zval = 34.94 TagRS{ kinds = RS_Key0_TagN bytes = 144720 } TagArray{ size = 919379 bytes = 3447680 } seq expand size = 6435653 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0323 GB dict = 1.08 MB all = 0.0388 GB } UnZip/Zip{ index = 10.4447 value = 3.3495 dict = 1.00 all = 3.4602 } Zip/UnZip{ index = 0.0957 value = 0.2986 dict = 1.00 all = 0.2890 } ---------------------------- total value len = 9.190617 GB avg = 0.167 KB (by entry num) total key len = 1.518833 GB avg = 0.028 KB total ukey len = 1.053601 GB avg = 0.015 KB total ukey num = 0.068336577 Billion total entry num = 0.055144306 Billion write speed all = 0.300325600 MB/sec (with seq num) write speed all = 0.287954285 MB/sec (without seq num) 2024/11/23-18:49:31.727081 139957542479424 [compaction_job.cc:1948] [default] [JOB 33516] Generated table #275728: 925521 keys, 38788774 bytes, temperature: kWarm 2024/11/23-18:49:31.727152 139957542479424 EVENT_LOG_v1 {"time_micros": 1732358971727115, "cf_name": "default", "job": 33516, "event": "table_file_creation", "file_number": 275728, "file_size": 38788774, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012775935, "table_properties": {"data_size": 32334016, "index_size": 1772224, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25914588, "raw_average_key_size": 28, "raw_value_size": 108303258, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925521, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358968, "oldest_key_time": 0, "file_creation_time": 1732358968, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275728, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:49:31.765205 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275729.sst) /dev/shm/ToplingTemp/Topling-155990-XrBh3o 2024/11/23-18:49:32.893250 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275729.sst): first pass time = 1.13's, 118.986'MB/sec 2024/11/23-18:49:34.332863 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275729.sst): old prealloc_size = 147639500, real_size = 35761600 2024/11/23-18:49:34.333591 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275729.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927954 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0338 GB avg-val = 116.64 avg-zval = 36.37 2024/11/23-18:49:34.334489 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275729.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:49:35.307703 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275729.sst): second pass time = 1.44's, 75.232'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.97's, 41.011'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2522.058'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.282'MB/sec write SST data time = 0.15's, 262.252'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34194.546'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.232'MB/sec zip pipeline throughput = 75.232'MB/sec entries = 927954 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.64 avg-zval = 36.37 usrkeys = 927954 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.64 avg-zval = 36.37 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927954 bytes = 3363856 } seq expand size = 6495678 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0338 GB dict = 1.08 MB all = 0.0400 GB } UnZip/Zip{ index = 10.4447 value = 3.2067 dict = 1.00 all = 3.3572 } Zip/UnZip{ index = 0.0957 value = 0.3118 dict = 1.00 all = 0.2979 } ---------------------------- total value len = 9.305347 GB avg = 0.166 KB (by entry num) total key len = 1.544816 GB avg = 0.028 KB total ukey len = 1.072160 GB avg = 0.015 KB total ukey num = 0.069264531 Billion total entry num = 0.056072260 Billion write speed all = 0.304240665 MB/sec (with seq num) write speed all = 0.291662450 MB/sec (without seq num) 2024/11/23-18:49:35.358765 139957542479424 [compaction_job.cc:1948] [default] [JOB 33516] Generated table #275729: 927954 keys, 39978900 bytes, temperature: kWarm 2024/11/23-18:49:35.358831 139957542479424 EVENT_LOG_v1 {"time_micros": 1732358975358797, "cf_name": "default", "job": 33516, "event": "table_file_creation", "file_number": 275729, "file_size": 39978900, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775967, "table_properties": {"data_size": 33752560, "index_size": 1776896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25982712, "raw_average_key_size": 28, "raw_value_size": 108235056, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927954, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358971, "oldest_key_time": 0, "file_creation_time": 1732358971, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275729, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:49:35.394215 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275730.sst) /dev/shm/ToplingTemp/Topling-155990-P5Zkyq 2024/11/23-18:49:36.513784 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275730.sst): first pass time = 1.12's, 119.887'MB/sec 2024/11/23-18:49:37.871382 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275730.sst): old prealloc_size = 147639500, real_size = 32747168 2024/11/23-18:49:37.872219 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275730.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928520 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0307 GB avg-val = 116.55 avg-zval = 33.12 2024/11/23-18:49:37.873193 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275730.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:49:38.900553 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275730.sst): second pass time = 1.36's, 79.770'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 36.032'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2191.456'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.267'MB/sec write SST data time = 0.16's, 231.299'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 37170.654'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.770'MB/sec zip pipeline throughput = 79.770'MB/sec entries = 928520 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.55 avg-zval = 33.12 usrkeys = 928520 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.55 avg-zval = 33.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928520 bytes = 3481968 } seq expand size = 6499640 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5143 value = 3.5195 dict = 1.00 all = 3.6194 } Zip/UnZip{ index = 0.0951 value = 0.2841 dict = 1.00 all = 0.2763 } ---------------------------- total value len = 9.420066 GB avg = 0.165 KB (by entry num) total key len = 1.570814 GB avg = 0.028 KB total ukey len = 1.090730 GB avg = 0.016 KB total ukey num = 0.070193051 Billion total entry num = 0.057000780 Billion write speed all = 0.308155366 MB/sec (with seq num) write speed all = 0.295370152 MB/sec (without seq num) 2024/11/23-18:49:38.947677 139957542479424 [compaction_job.cc:1948] [default] [JOB 33516] Generated table #275730: 928520 keys, 37082464 bytes, temperature: kWarm 2024/11/23-18:49:38.947747 139957542479424 EVENT_LOG_v1 {"time_micros": 1732358978947710, "cf_name": "default", "job": 33516, "event": "table_file_creation", "file_number": 275730, "file_size": 37082464, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 4012776021, "table_properties": {"data_size": 30748624, "index_size": 1766208, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25998560, "raw_average_key_size": 28, "raw_value_size": 108219308, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928520, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358975, "oldest_key_time": 0, "file_creation_time": 1732358975, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275730, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:49:38.984538 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275731.sst) /dev/shm/ToplingTemp/Topling-155990-KZ8exq 2024/11/23-18:49:40.228705 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275731.sst): first pass time = 1.24's, 107.881'MB/sec 2024/11/23-18:49:41.759370 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275731.sst): old prealloc_size = 147639500, real_size = 34539904 2024/11/23-18:49:41.760099 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275731.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927343 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0325 GB avg-val = 116.73 avg-zval = 35.08 2024/11/23-18:49:41.760875 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275731.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:49:42.639693 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275731.sst): second pass time = 1.53's, 70.767'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 43.547'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2519.961'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.16's, 245.493'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 29956.799'MB/sec zip my value time = 1.53's, unzip length = 0.108'GB zip my value throughput = 70.767'MB/sec zip pipeline throughput = 70.767'MB/sec entries = 927343 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.73 avg-zval = 35.08 usrkeys = 927343 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.35 avg-zval = 35.08 TagRS{ kinds = RS_Key0_TagN bytes = 145000 } TagArray{ size = 744346 bytes = 2791312 } seq expand size = 5210422 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.4457 value = 3.3275 dict = 1.00 all = 3.5012 } Zip/UnZip{ index = 0.0957 value = 0.3005 dict = 1.00 all = 0.2856 } ---------------------------- total value len = 9.533529 GB avg = 0.165 KB (by entry num) total key len = 1.596780 GB avg = 0.028 KB total ukey len = 1.109277 GB avg = 0.016 KB total ukey num = 0.071120394 Billion total entry num = 0.057928123 Billion write speed all = 0.312031855 MB/sec (with seq num) write speed all = 0.299040000 MB/sec (without seq num) 2024/11/23-18:49:42.688466 139957542479424 [compaction_job.cc:1948] [default] [JOB 33516] Generated table #275731: 927343 keys, 38334330 bytes, temperature: kWarm 2024/11/23-18:49:42.688534 139957542479424 EVENT_LOG_v1 {"time_micros": 1732358982688497, "cf_name": "default", "job": 33516, "event": "table_file_creation", "file_number": 275731, "file_size": 38334330, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776210, "table_properties": {"data_size": 32532336, "index_size": 1775552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25965604, "raw_average_key_size": 28, "raw_value_size": 108252190, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927343, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358978, "oldest_key_time": 0, "file_creation_time": 1732358978, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275731, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:49:42.726177 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275732.sst) /dev/shm/ToplingTemp/Topling-155990-qZmWhp 2024/11/23-18:49:43.773251 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275732.sst): first pass time = 1.05's, 128.192'MB/sec 2024/11/23-18:49:45.076849 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275732.sst): old prealloc_size = 147639500, real_size = 31545296 2024/11/23-18:49:45.077415 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275732.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933457 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0295 GB avg-val = 115.79 avg-zval = 31.64 2024/11/23-18:49:45.078218 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275732.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:49:45.922925 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275732.sst): second pass time = 1.30's, 82.958'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 42.290'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3271.509'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.13's, 284.674'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36799.249'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 82.958'MB/sec zip pipeline throughput = 82.958'MB/sec entries = 933457 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.79 avg-zval = 31.64 usrkeys = 933457 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.79 avg-zval = 31.64 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933457 bytes = 3383792 } seq expand size = 6534199 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.08 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5138 value = 3.6593 dict = 1.00 all = 3.7513 } Zip/UnZip{ index = 0.0951 value = 0.2733 dict = 1.00 all = 0.2666 } ---------------------------- total value len = 9.648144 GB avg = 0.164 KB (by entry num) total key len = 1.622917 GB avg = 0.028 KB total ukey len = 1.127946 GB avg = 0.016 KB total ukey num = 0.072053851 Billion total entry num = 0.058861580 Billion write speed all = 0.315948677 MB/sec (with seq num) write speed all = 0.302748685 MB/sec (without seq num) 2024/11/23-18:49:45.971881 139957542479424 [compaction_job.cc:1948] [default] [JOB 33516] Generated table #275732: 933457 keys, 35779434 bytes, temperature: kWarm 2024/11/23-18:49:45.971964 139957542479424 EVENT_LOG_v1 {"time_micros": 1732358985971920, "cf_name": "default", "job": 33516, "event": "table_file_creation", "file_number": 275732, "file_size": 35779434, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776228, "table_properties": {"data_size": 29536064, "index_size": 1775680, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26136796, "raw_average_key_size": 28, "raw_value_size": 108081050, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933457, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358982, "oldest_key_time": 0, "file_creation_time": 1732358982, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275732, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:49:46.009321 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275733.sst) /dev/shm/ToplingTemp/Topling-155990-M7t4As 2024/11/23-18:49:46.944274 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275733.sst): first pass time = 0.93's, 143.561'MB/sec 2024/11/23-18:49:48.262528 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275733.sst): old prealloc_size = 147639500, real_size = 33225760 2024/11/23-18:49:48.263111 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275733.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931033 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0312 GB avg-val = 116.16 avg-zval = 33.51 2024/11/23-18:49:48.263889 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275733.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:49:49.104256 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275733.sst): second pass time = 1.32's, 82.084'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.496'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3197.878'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.325'MB/sec write SST data time = 0.12's, 299.718'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 26460.157'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.084'MB/sec zip pipeline throughput = 82.084'MB/sec entries = 931033 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.16 avg-zval = 33.51 usrkeys = 931033 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.16 avg-zval = 33.51 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931033 bytes = 3375008 } seq expand size = 6517231 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.3784 value = 3.4665 dict = 1.00 all = 3.5837 } Zip/UnZip{ index = 0.0964 value = 0.2885 dict = 1.00 all = 0.2790 } ---------------------------- total value len = 9.762810 GB avg = 0.163 KB (by entry num) total key len = 1.648986 GB avg = 0.028 KB total ukey len = 1.146567 GB avg = 0.016 KB total ukey num = 0.072984884 Billion total entry num = 0.059792613 Billion write speed all = 0.319865217 MB/sec (with seq num) write speed all = 0.306457632 MB/sec (without seq num) 2024/11/23-18:49:49.154561 139957542479424 [compaction_job.cc:1948] [default] [JOB 33516] Generated table #275733: 931033 keys, 37452420 bytes, temperature: kWarm 2024/11/23-18:49:49.154631 139957542479424 EVENT_LOG_v1 {"time_micros": 1732358989154595, "cf_name": "default", "job": 33516, "event": "table_file_creation", "file_number": 275733, "file_size": 37452420, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776268, "table_properties": {"data_size": 31198672, "index_size": 1794176, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26068924, "raw_average_key_size": 28, "raw_value_size": 108148938, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931033, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358985, "oldest_key_time": 0, "file_creation_time": 1732358985, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275733, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:49:49.190278 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275734.sst) /dev/shm/ToplingTemp/Topling-155990-OG1Nzr 2024/11/23-18:49:50.227726 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275734.sst): first pass time = 1.04's, 129.378'MB/sec 2024/11/23-18:49:51.565932 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275734.sst): old prealloc_size = 147639500, real_size = 34711680 2024/11/23-18:49:51.566519 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275734.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925254 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0327 GB avg-val = 117.06 avg-zval = 35.35 2024/11/23-18:49:51.567327 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275734.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:49:52.412360 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275734.sst): second pass time = 1.34's, 80.988'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 46.120'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3145.902'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.12's, 313.397'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 27213.886'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.988'MB/sec zip pipeline throughput = 80.988'MB/sec entries = 925254 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.06 avg-zval = 35.35 usrkeys = 925254 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.06 avg-zval = 35.35 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925254 bytes = 3469712 } seq expand size = 6476778 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0327 GB dict = 1.08 MB all = 0.0390 GB } UnZip/Zip{ index = 10.4425 value = 3.3114 dict = 1.00 all = 3.4383 } Zip/UnZip{ index = 0.0958 value = 0.3020 dict = 1.00 all = 0.2908 } ---------------------------- total value len = 9.877598 GB avg = 0.163 KB (by entry num) total key len = 1.674893 GB avg = 0.028 KB total ukey len = 1.165072 GB avg = 0.016 KB total ukey num = 0.073910138 Billion total entry num = 0.060717867 Billion write speed all = 0.323778771 MB/sec (with seq num) write speed all = 0.310164975 MB/sec (without seq num) 2024/11/23-18:49:52.465519 139957542479424 [compaction_job.cc:1948] [default] [JOB 33516] Generated table #275734: 925254 keys, 39036148 bytes, temperature: kWarm 2024/11/23-18:49:52.465591 139957542479424 EVENT_LOG_v1 {"time_micros": 1732358992465553, "cf_name": "default", "job": 33516, "event": "table_file_creation", "file_number": 275734, "file_size": 39036148, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012774796, "table_properties": {"data_size": 32708080, "index_size": 1772096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25907112, "raw_average_key_size": 28, "raw_value_size": 108310652, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925254, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358989, "oldest_key_time": 0, "file_creation_time": 1732358989, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275734, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:49:52.506363 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275735.sst) /dev/shm/ToplingTemp/Topling-155990-KwQ9Wr 2024/11/23-18:49:53.392576 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275735.sst): first pass time = 0.89's, 151.458'MB/sec 2024/11/23-18:49:54.720530 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275735.sst): old prealloc_size = 147639500, real_size = 32473280 2024/11/23-18:49:54.721146 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275735.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933317 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0305 GB avg-val = 115.81 avg-zval = 32.64 2024/11/23-18:49:54.721968 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275735.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:49:55.582076 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275735.sst): second pass time = 1.33's, 81.439'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.607'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2991.936'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 287.889'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35876.868'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.439'MB/sec zip pipeline throughput = 81.439'MB/sec entries = 933317 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.81 avg-zval = 32.64 usrkeys = 933317 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.81 avg-zval = 32.64 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933317 bytes = 3383296 } seq expand size = 6533219 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5137 value = 3.5479 dict = 1.00 all = 3.6564 } Zip/UnZip{ index = 0.0951 value = 0.2819 dict = 1.00 all = 0.2735 } ---------------------------- total value len = 9.992216 GB avg = 0.162 KB (by entry num) total key len = 1.701026 GB avg = 0.028 KB total ukey len = 1.183738 GB avg = 0.016 KB total ukey num = 0.074843455 Billion total entry num = 0.061651184 Billion write speed all = 0.327694455 MB/sec (with seq num) write speed all = 0.313872624 MB/sec (without seq num) 2024/11/23-18:49:55.630673 139957542479424 [compaction_job.cc:1948] [default] [JOB 33516] Generated table #275735: 933317 keys, 36707264 bytes, temperature: kWarm 2024/11/23-18:49:55.630744 139957542479424 EVENT_LOG_v1 {"time_micros": 1732358995630707, "cf_name": "default", "job": 33516, "event": "table_file_creation", "file_number": 275735, "file_size": 36707264, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776316, "table_properties": {"data_size": 30464368, "index_size": 1775424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26132876, "raw_average_key_size": 28, "raw_value_size": 108084974, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933317, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358992, "oldest_key_time": 0, "file_creation_time": 1732358992, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275735, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:49:55.672322 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275736.sst) /dev/shm/ToplingTemp/Topling-155990-8T7i5r 2024/11/23-18:49:56.636472 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275736.sst): first pass time = 0.96's, 139.214'MB/sec 2024/11/23-18:49:57.945114 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275736.sst): old prealloc_size = 147639500, real_size = 33961568 2024/11/23-18:49:57.945655 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275736.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923008 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0320 GB avg-val = 117.41 avg-zval = 34.64 2024/11/23-18:49:57.946739 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275736.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:49:58.794696 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275736.sst): second pass time = 1.31's, 82.865'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.059'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3388.761'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 299.628'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33864.315'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.865'MB/sec zip pipeline throughput = 82.865'MB/sec entries = 923008 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.41 avg-zval = 34.64 usrkeys = 923008 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.41 avg-zval = 34.64 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923008 bytes = 3461296 } seq expand size = 6461056 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.5128 value = 3.3894 dict = 1.00 all = 3.5062 } Zip/UnZip{ index = 0.0951 value = 0.2950 dict = 1.00 all = 0.2852 } ---------------------------- total value len = 10.107051 GB avg = 0.162 KB (by entry num) total key len = 1.726870 GB avg = 0.028 KB total ukey len = 1.202199 GB avg = 0.016 KB total ukey num = 0.075766463 Billion total entry num = 0.062574192 Billion write speed all = 0.331607021 MB/sec (with seq num) write speed all = 0.317579520 MB/sec (without seq num) 2024/11/23-18:49:58.845319 139957542479424 [compaction_job.cc:1948] [default] [JOB 33516] Generated table #275736: 923008 keys, 38280580 bytes, temperature: kWarm 2024/11/23-18:49:58.845390 139957542479424 EVENT_LOG_v1 {"time_micros": 1732358998845352, "cf_name": "default", "job": 33516, "event": "table_file_creation", "file_number": 275736, "file_size": 38280580, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776344, "table_properties": {"data_size": 31974672, "index_size": 1755968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25844224, "raw_average_key_size": 28, "raw_value_size": 108373600, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923008, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358995, "oldest_key_time": 0, "file_creation_time": 1732358995, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275736, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:49:58.884838 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275737.sst) /dev/shm/ToplingTemp/Topling-155990-fhRIpr 2024/11/23-18:49:59.762944 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275737.sst): first pass time = 0.88's, 152.859'MB/sec 2024/11/23-18:50:01.085332 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275737.sst): old prealloc_size = 147639500, real_size = 33058960 2024/11/23-18:50:01.085891 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275737.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928698 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.52 avg-zval = 33.43 2024/11/23-18:50:01.088204 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275737.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:50:01.947777 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275737.sst): second pass time = 1.32's, 81.917'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.360'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3329.008'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.316'MB/sec write SST data time = 0.13's, 294.049'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35246.735'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.917'MB/sec zip pipeline throughput = 81.917'MB/sec entries = 928698 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.52 avg-zval = 33.43 usrkeys = 928698 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.52 avg-zval = 33.43 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928698 bytes = 3482640 } seq expand size = 6500886 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4436 value = 3.4854 dict = 1.00 all = 3.5892 } Zip/UnZip{ index = 0.0958 value = 0.2869 dict = 1.00 all = 0.2786 } ---------------------------- total value len = 10.221766 GB avg = 0.161 KB (by entry num) total key len = 1.752873 GB avg = 0.028 KB total ukey len = 1.220773 GB avg = 0.016 KB total ukey num = 0.076695161 Billion total entry num = 0.063502890 Billion write speed all = 0.335520560 MB/sec (with seq num) write speed all = 0.321286127 MB/sec (without seq num) 2024/11/23-18:50:01.996416 139957542479424 [compaction_job.cc:1948] [default] [JOB 33516] Generated table #275737: 928698 keys, 37394676 bytes, temperature: kWarm 2024/11/23-18:50:01.996499 139957542479424 EVENT_LOG_v1 {"time_micros": 1732359001996451, "cf_name": "default", "job": 33516, "event": "table_file_creation", "file_number": 275737, "file_size": 37394676, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012771944, "table_properties": {"data_size": 31048128, "index_size": 1778496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26003544, "raw_average_key_size": 28, "raw_value_size": 108214320, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928698, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358998, "oldest_key_time": 0, "file_creation_time": 1732358998, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275737, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:50:02.040461 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275738.sst) /dev/shm/ToplingTemp/Topling-155990-DiWWDs 2024/11/23-18:50:02.871262 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275738.sst): first pass time = 0.83's, 161.563'MB/sec 2024/11/23-18:50:04.282935 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275738.sst): old prealloc_size = 147639500, real_size = 35323520 2024/11/23-18:50:04.283519 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275738.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929245 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0333 GB avg-val = 116.44 avg-zval = 35.85 2024/11/23-18:50:04.283949 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275738.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:50:05.050049 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275738.sst): second pass time = 1.41's, 76.686'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.77's, 49.209'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3294.369'MB/sec (index lex order gen) rebuild zvType time = 0.64's, 0.363'MB/sec write SST data time = 0.13's, 296.893'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32066.479'MB/sec zip my value time = 1.41's, unzip length = 0.108'GB zip my value throughput = 76.686'MB/sec zip pipeline throughput = 76.686'MB/sec entries = 929245 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.44 avg-zval = 35.85 usrkeys = 929245 avg-key = 20.00 avg-zkey = 1.92 avg-val = 119.49 avg-zval = 35.85 TagRS{ kinds = RS_Key0_TagN bytes = 145280 } TagArray{ size = 405779 bytes = 1420240 } seq expand size = 2840453 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0333 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.4430 value = 3.2481 dict = 1.00 all = 3.5557 } Zip/UnZip{ index = 0.0958 value = 0.3079 dict = 1.00 all = 0.2812 } ---------------------------- total value len = 10.332805 GB avg = 0.160 KB (by entry num) total key len = 1.778892 GB avg = 0.028 KB total ukey len = 1.239357 GB avg = 0.016 KB total ukey num = 0.077624406 Billion total entry num = 0.064432135 Billion write speed all = 0.339331335 MB/sec (with seq num) write speed all = 0.324889863 MB/sec (without seq num) 2024/11/23-18:50:05.097634 139957542479424 [compaction_job.cc:1948] [default] [JOB 33516] Generated table #275738: 929245 keys, 37746797 bytes, temperature: kWarm 2024/11/23-18:50:05.097703 139957542479424 EVENT_LOG_v1 {"time_micros": 1732359005097666, "cf_name": "default", "job": 33516, "event": "table_file_creation", "file_number": 275738, "file_size": 37746797, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 33311408, "index_size": 1779648, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26018860, "raw_average_key_size": 28, "raw_value_size": 108198914, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929245, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732359002, "oldest_key_time": 0, "file_creation_time": 1732359002, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275738, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:50:05.136339 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275739.sst) /dev/shm/ToplingTemp/Topling-155990-L28PUo 2024/11/23-18:50:06.141588 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275739.sst): first pass time = 1.01's, 133.523'MB/sec 2024/11/23-18:50:07.668676 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275739.sst): old prealloc_size = 147639500, real_size = 36881840 2024/11/23-18:50:07.669255 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275739.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926813 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0349 GB avg-val = 116.82 avg-zval = 37.63 2024/11/23-18:50:07.669486 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275739.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:50:08.380107 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275739.sst): second pass time = 1.53's, 70.946'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.71's, 53.932'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3168.240'MB/sec (index lex order gen) rebuild zvType time = 0.59's, 0.395'MB/sec write SST data time = 0.12's, 309.119'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32465.238'MB/sec zip my value time = 1.53's, unzip length = 0.108'GB zip my value throughput = 70.946'MB/sec zip pipeline throughput = 70.946'MB/sec entries = 926813 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.82 avg-zval = 37.63 usrkeys = 926813 avg-key = 20.00 avg-zkey = 1.91 avg-val = 117.98 avg-zval = 37.63 TagRS{ kinds = RS_Key0_TagN bytes = 144920 } TagArray{ size = 154398 bytes = 482512 } seq expand size = 1080786 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0349 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.4457 value = 3.1044 dict = 1.00 all = 3.4982 } Zip/UnZip{ index = 0.0957 value = 0.3221 dict = 1.00 all = 0.2859 } ---------------------------- total value len = 10.442153 GB avg = 0.160 KB (by entry num) total key len = 1.804843 GB avg = 0.028 KB total ukey len = 1.257894 GB avg = 0.016 KB total ukey num = 0.078551219 Billion total entry num = 0.065358948 Billion write speed all = 0.343089963 MB/sec (with seq num) write speed all = 0.328442127 MB/sec (without seq num) 2024/11/23-18:50:08.431274 139957542479424 [compaction_job.cc:1948] [default] [JOB 33516] Generated table #275739: 926813 keys, 38367269 bytes, temperature: kWarm 2024/11/23-18:50:08.431344 139957542479424 EVENT_LOG_v1 {"time_micros": 1732359008431308, "cf_name": "default", "job": 33516, "event": "table_file_creation", "file_number": 275739, "file_size": 38367269, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700812, "table_properties": {"data_size": 34875424, "index_size": 1774528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25950764, "raw_average_key_size": 28, "raw_value_size": 108267054, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926813, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732359005, "oldest_key_time": 0, "file_creation_time": 1732359005, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275739, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:50:08.467400 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275740.sst) /dev/shm/ToplingTemp/Topling-155990-lxu6vp 2024/11/23-18:50:09.464149 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275740.sst): first pass time = 1.00's, 134.662'MB/sec 2024/11/23-18:50:10.788331 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275740.sst): old prealloc_size = 147639500, real_size = 35264304 2024/11/23-18:50:10.788884 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275740.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923965 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0333 GB avg-val = 117.26 avg-zval = 36.03 2024/11/23-18:50:10.791013 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275740.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:50:11.630720 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275740.sst): second pass time = 1.32's, 81.869'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.872'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3319.177'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 298.488'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 34700.285'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.869'MB/sec zip pipeline throughput = 81.869'MB/sec entries = 923965 avg-key = 28.00 avg-zkey = 1.89 avg-val = 117.26 avg-zval = 36.03 usrkeys = 923965 avg-key = 20.00 avg-zkey = 1.89 avg-val = 124.22 avg-zval = 36.03 TagRS{ kinds = RS_Key0_TagN bytes = 144480 } TagArray{ size = 918253 bytes = 3213904 } seq expand size = 6427771 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0333 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.5827 value = 3.2549 dict = 1.00 all = 3.3993 } Zip/UnZip{ index = 0.0945 value = 0.3072 dict = 1.00 all = 0.2942 } ---------------------------- total value len = 10.556928 GB avg = 0.159 KB (by entry num) total key len = 1.830714 GB avg = 0.028 KB total ukey len = 1.276373 GB avg = 0.016 KB total ukey num = 0.079475184 Billion total entry num = 0.066282913 Billion write speed all = 0.346998440 MB/sec (with seq num) write speed all = 0.332144883 MB/sec (without seq num) 2024/11/23-18:50:11.680547 139957542479424 [compaction_job.cc:1948] [default] [JOB 33516] Generated table #275740: 923965 keys, 39483638 bytes, temperature: kWarm 2024/11/23-18:50:11.680620 139957542479424 EVENT_LOG_v1 {"time_micros": 1732359011680581, "cf_name": "default", "job": 33516, "event": "table_file_creation", "file_number": 275740, "file_size": 39483638, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4195147145, "table_properties": {"data_size": 33286944, "index_size": 1746176, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25871020, "raw_average_key_size": 28, "raw_value_size": 108346790, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923965, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732359008, "oldest_key_time": 0, "file_creation_time": 1732359008, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275740, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:50:11.721933 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275741.sst) /dev/shm/ToplingTemp/Topling-155990-fKjqxs 2024/11/23-18:50:12.601712 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275741.sst): first pass time = 0.88's, 152.566'MB/sec 2024/11/23-18:50:13.942617 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275741.sst): old prealloc_size = 147639500, real_size = 35104240 2024/11/23-18:50:13.943225 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275741.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929468 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0331 GB avg-val = 116.40 avg-zval = 35.62 2024/11/23-18:50:13.945120 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275741.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:50:14.778415 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275741.sst): second pass time = 1.34's, 80.763'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.637'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3025.817'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.329'MB/sec write SST data time = 0.13's, 303.349'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 28324.761'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.763'MB/sec zip pipeline throughput = 80.763'MB/sec entries = 929468 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.40 avg-zval = 35.62 usrkeys = 929468 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.40 avg-zval = 35.62 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929468 bytes = 3020784 } seq expand size = 6506276 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0331 GB dict = 1.08 MB all = 0.0390 GB } UnZip/Zip{ index = 10.5121 value = 3.2683 dict = 1.00 all = 3.4434 } Zip/UnZip{ index = 0.0951 value = 0.3060 dict = 1.00 all = 0.2904 } ---------------------------- total value len = 10.671627 GB avg = 0.159 KB (by entry num) total key len = 1.856739 GB avg = 0.028 KB total ukey len = 1.294962 GB avg = 0.016 KB total ukey num = 0.080404652 Billion total entry num = 0.067212381 Billion write speed all = 0.350909416 MB/sec (with seq num) write speed all = 0.335848900 MB/sec (without seq num) 2024/11/23-18:50:14.830630 139957542479424 [compaction_job.cc:1948] [default] [JOB 33516] Generated table #275741: 929468 keys, 38977912 bytes, temperature: kWarm 2024/11/23-18:50:14.830704 139957542479424 EVENT_LOG_v1 {"time_micros": 1732359014830668, "cf_name": "default", "job": 33516, "event": "table_file_creation", "file_number": 275741, "file_size": 38977912, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4138852857, "largest_seqno": 4195147215, "table_properties": {"data_size": 33103328, "index_size": 1768384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26025104, "raw_average_key_size": 28, "raw_value_size": 108192760, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929468, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732359011, "oldest_key_time": 0, "file_creation_time": 1732359011, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275741, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:50:14.874464 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275742.sst) /dev/shm/ToplingTemp/Topling-155990-uVQ1Rs 2024/11/23-18:50:15.853402 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275742.sst): first pass time = 0.98's, 137.110'MB/sec 2024/11/23-18:50:17.166623 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275742.sst): old prealloc_size = 147639500, real_size = 33691472 2024/11/23-18:50:17.167212 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275742.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926109 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0317 GB avg-val = 116.93 avg-zval = 34.23 2024/11/23-18:50:17.167990 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275742.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:50:18.016693 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275742.sst): second pass time = 1.31's, 82.512'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.318'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3125.912'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 297.150'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34236.028'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.512'MB/sec zip pipeline throughput = 82.512'MB/sec entries = 926109 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.93 avg-zval = 34.23 usrkeys = 926109 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.93 avg-zval = 34.23 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926109 bytes = 3125632 } seq expand size = 6482763 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5133 value = 3.4162 dict = 1.00 all = 3.5628 } Zip/UnZip{ index = 0.0951 value = 0.2927 dict = 1.00 all = 0.2807 } ---------------------------- total value len = 10.786396 GB avg = 0.158 KB (by entry num) total key len = 1.882670 GB avg = 0.028 KB total ukey len = 1.313485 GB avg = 0.016 KB total ukey num = 0.081330761 Billion total entry num = 0.068138490 Billion write speed all = 0.354818141 MB/sec (with seq num) write speed all = 0.339551493 MB/sec (without seq num) 2024/11/23-18:50:18.064595 139957542479424 [compaction_job.cc:1948] [default] [JOB 33516] Generated table #275742: 926109 keys, 37672066 bytes, temperature: kWarm 2024/11/23-18:50:18.064669 139957542479424 EVENT_LOG_v1 {"time_micros": 1732359018064630, "cf_name": "default", "job": 33516, "event": "table_file_creation", "file_number": 275742, "file_size": 37672066, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4096159344, "largest_seqno": 4195146672, "table_properties": {"data_size": 31697984, "index_size": 1761792, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25931052, "raw_average_key_size": 28, "raw_value_size": 108286690, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926109, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732359014, "oldest_key_time": 0, "file_creation_time": 1732359014, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275742, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:50:18.107020 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275743.sst) /dev/shm/ToplingTemp/Topling-155990-t6RQ7q 2024/11/23-18:50:19.001013 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275743.sst): first pass time = 0.89's, 150.139'MB/sec 2024/11/23-18:50:20.324843 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275743.sst): old prealloc_size = 147639500, real_size = 33919968 2024/11/23-18:50:20.325460 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275743.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933344 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0319 GB avg-val = 115.80 avg-zval = 34.18 2024/11/23-18:50:20.326231 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275743.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:50:21.169207 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275743.sst): second pass time = 1.32's, 81.692'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.779'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3011.840'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.326'MB/sec write SST data time = 0.13's, 296.370'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34388.047'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.692'MB/sec zip pipeline throughput = 81.692'MB/sec entries = 933344 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.80 avg-zval = 34.18 usrkeys = 933344 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.80 avg-zval = 34.18 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933344 bytes = 3033376 } seq expand size = 6533408 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.4455 value = 3.3883 dict = 1.00 all = 3.5500 } Zip/UnZip{ index = 0.0957 value = 0.2951 dict = 1.00 all = 0.2817 } ---------------------------- total value len = 10.901014 GB avg = 0.158 KB (by entry num) total key len = 1.908804 GB avg = 0.028 KB total ukey len = 1.332151 GB avg = 0.016 KB total ukey num = 0.082264105 Billion total entry num = 0.069071834 Billion write speed all = 0.358728437 MB/sec (with seq num) write speed all = 0.343254037 MB/sec (without seq num) 2024/11/23-18:50:21.219200 139957542479424 [compaction_job.cc:1948] [default] [JOB 33516] Generated table #275743: 933344 keys, 37807922 bytes, temperature: kWarm 2024/11/23-18:50:21.219273 139957542479424 EVENT_LOG_v1 {"time_micros": 1732359021219237, "cf_name": "default", "job": 33516, "event": "table_file_creation", "file_number": 275743, "file_size": 37807922, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4137892167, "largest_seqno": 4195146335, "table_properties": {"data_size": 31899408, "index_size": 1787072, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26133632, "raw_average_key_size": 28, "raw_value_size": 108084104, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933344, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732359018, "oldest_key_time": 0, "file_creation_time": 1732359018, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275743, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:50:21.262783 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275744.sst) /dev/shm/ToplingTemp/Topling-155990-mRqC3s 2024/11/23-18:50:22.205602 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275744.sst): first pass time = 0.94's, 142.364'MB/sec 2024/11/23-18:50:23.500525 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275744.sst): old prealloc_size = 147639500, real_size = 33227664 2024/11/23-18:50:23.501125 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275744.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 935507 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0312 GB avg-val = 115.47 avg-zval = 33.37 2024/11/23-18:50:23.503093 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275744.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:50:24.353526 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275744.sst): second pass time = 1.29's, 83.468'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.653'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3066.344'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 289.352'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 23522.211'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.468'MB/sec zip pipeline throughput = 83.468'MB/sec entries = 935507 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.47 avg-zval = 33.37 usrkeys = 935507 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.47 avg-zval = 33.37 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 935507 bytes = 3157344 } seq expand size = 6548549 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5134 value = 3.4607 dict = 1.00 all = 3.6047 } Zip/UnZip{ index = 0.0951 value = 0.2890 dict = 1.00 all = 0.2774 } ---------------------------- total value len = 11.015586 GB avg = 0.157 KB (by entry num) total key len = 1.934998 GB avg = 0.028 KB total ukey len = 1.350862 GB avg = 0.016 KB total ukey num = 0.083199612 Billion total entry num = 0.070007341 Billion write speed all = 0.362638147 MB/sec (with seq num) write speed all = 0.346955560 MB/sec (without seq num) 2024/11/23-18:50:24.400795 139957542479424 [compaction_job.cc:1948] [default] [JOB 33516] Generated table #275744: 935507 keys, 37234584 bytes, temperature: kWarm 2024/11/23-18:50:24.400868 139957542479424 EVENT_LOG_v1 {"time_micros": 1732359024400831, "cf_name": "default", "job": 33516, "event": "table_file_creation", "file_number": 275744, "file_size": 37234584, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4124223986, "largest_seqno": 4195147012, "table_properties": {"data_size": 31213952, "index_size": 1779648, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26194196, "raw_average_key_size": 28, "raw_value_size": 108023666, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 935507, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732359021, "oldest_key_time": 0, "file_creation_time": 1732359021, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275744, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:50:24.441160 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275745.sst) /dev/shm/ToplingTemp/Topling-155990-rYqxwp 2024/11/23-18:50:24.538303 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275745.sst): first pass time = 0.10's, 239.061'MB/sec 2024/11/23-18:50:24.752910 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275745.sst): old prealloc_size = 147639500, real_size = 6840816 2024/11/23-18:50:24.753083 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275745.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 160365 prefix = 4 raw-key = 0.0026 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0187 GB zip-val = 0.0065 GB avg-val = 116.74 avg-zval = 40.50 2024/11/23-18:50:24.753252 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275745.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0004 GB, waited 0.000 sec, Key+Value = 0.023 GB 2024/11/23-18:50:24.894059 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00/275745.sst): second pass time = 0.21's, 87.501'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.14's, 53.007'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2023.650'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.340'MB/sec write SST data time = 0.02's, 325.516'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.197'MB, throughput = 6502.794'MB/sec zip my value time = 0.21's, unzip length = 0.019'GB zip my value throughput = 87.501'MB/sec zip pipeline throughput = 87.501'MB/sec entries = 160365 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.74 avg-zval = 40.50 usrkeys = 160365 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.74 avg-zval = 40.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 160365 bytes = 481104 } seq expand size = 1122555 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0032 GB value = 0.0187 GB dict = 0.20 MB all = 0.0232 GB } __ZipSize{ index = 0.0003 GB value = 0.0065 GB dict = 0.20 MB all = 0.0075 GB } UnZip/Zip{ index = 10.4644 value = 2.8827 dict = 1.00 all = 3.1028 } Zip/UnZip{ index = 0.0956 value = 0.3469 dict = 1.00 all = 0.3223 } ---------------------------- total value len = 11.035429 GB avg = 0.157 KB (by entry num) total key len = 1.939488 GB avg = 0.028 KB total ukey len = 1.354069 GB avg = 0.016 KB total ukey num = 0.083359977 Billion total entry num = 0.070167706 Billion write speed all = 0.363314026 MB/sec (with seq num) write speed all = 0.347595753 MB/sec (without seq num) 2024/11/23-18:50:24.910111 139957542479424 [compaction_job.cc:1948] [default] [JOB 33516] Generated table #275745: 160365 keys, 7480519 bytes, temperature: kWarm 2024/11/23-18:50:24.910178 139957542479424 EVENT_LOG_v1 {"time_micros": 1732359024910144, "cf_name": "default", "job": 33516, "event": "table_file_creation", "file_number": 275745, "file_size": 7480519, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4179164037, "largest_seqno": 4195145983, "table_properties": {"data_size": 6494048, "index_size": 306496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4490220, "raw_average_key_size": 28, "raw_value_size": 18720690, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 160365, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732359024, "oldest_key_time": 0, "file_creation_time": 1732359024, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275745, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:50:24.929159 139957542479424 [compaction_job.cc:703] job-33516: subcompact[0], size: 0.732079 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33516/att-00 [275726,275727,275728,275729,275730,275731,275732,275733,275734,275735,275736,275737,275738,275739,275740,275741,275742,275743,275744,275745] 2024/11/23-18:50:25.049373 139957542479424 (Original Log Time 2024/11/23-18:49:20.361529) EVENT_LOG_v1 {"time_micros": 1732358960361467, "job": 33516, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [275699], "files_L3": [275640, 275641, 275642, 275643, 275644, 275645, 275646, 275647, 275648, 275649, 275650, 275651, 275652, 275653, 275654, 275655, 275656, 275662, 275720], "score": 0.997712, "input_data_size": 751529060, "oldest_snapshot_seqno": 4195340871} 2024/11/23-18:50:25.049377 139957542479424 (Original Log Time 2024/11/23-18:50:25.047647) [compaction_job.cc:2022] [default] [JOB 33516] Compacted 1@2 + 19@3 files to L3 => 732078999 bytes 2024/11/23-18:50:25.049378 139957542479424 (Original Log Time 2024/11/23-18:50:25.049303) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 20 0 0 0] max score 0.03, MB/sec: 11.6 rd, 11.3 wr, level 3, files in(1, 19) out(20 +0 blob) MB in(27.1, 689.6 +0.0 blob) out(698.2 +0.0 blob), read-write-amplify(52.2) write-amplify(25.8) OK, records in: 17882219, records dropped: 72677 output_compression: Snappy 2024/11/23-18:50:25.049381 139957542479424 (Original Log Time 2024/11/23-18:50:25.049347) EVENT_LOG_v1 {"time_micros": 1732359025049327, "cf": "default", "job": 33516, "event": "compaction_finished", "compaction_time_micros": 64567638, "compaction_time_cpu_micros": 28077987, "output_level": 3, "num_output_files": 20, "total_output_size": 732078999, "num_input_records": 17882219, "num_output_records": 17809542, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 538396621, "file_range_sync_nanos": 0, "file_fsync_nanos": 14616, "file_prepare_write_nanos": 59964, "lsm_state": [0, 0, 0, 20, 0, 0, 0]} 2024/11/23-18:50:25.185067 139957542479424 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732358960,"executesMs":64849,"compactionJobId":33516,"attempt":0,"compactionInputRawBytes":2583669478,"compactionInputZipBytes":751529060,"compactionOutputRawBytes":0,"compactionOutputZipBytes":732078999,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-264cdb5290f52b2f9af27d2b58be4fbb-d83acf96a2a37396-00"}}, response = 2024/11/23-18:50:25.185154 139957542479424 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-33516/att-00: olev 3, work 64.849 s, result 7.817 ms, install 1.749 ms, input{raw 2.406 GiB zip 716.714 MiB}