2024/11/22-18:41:44.414381 140024802182720 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00: fork to child time = 0.028511 sec 2024/11/22-18:41:44.415470 140024802182720 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00 2024/11/22-18:41:44.425523 140024802182720 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00 2024/11/22-18:41:44.427292 140024802182720 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-18:41:44.427638 140024802182720 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-18:41:44.471465 140024802182720 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-07814/att-00/MANIFEST-206347 2024/11/22-18:41:44.485994 140024802182720 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-07814/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 221083, last_sequence is 3473491342, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 220932 2024/11/22-18:41:44.486005 140024802182720 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-18:41:44.486155 140024802182720 [version_set.cc:5721] Creating manifest 221083 2024/11/22-18:41:44.521720 140024802182720 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-18:41:44.544436 140024802182720 [compaction_job.cc:2437] [default] [JOB 7814] Compacting 1@2 + 13@3 files to L3, score 0.95, subcompactions 1 : 1 2024/11/22-18:41:44.544463 140024802182720 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 5936 Base level 2, inputs: [221076(27MB)], [221028(35MB) 221029(5794KB) 221006(35MB) 221007(36MB) 221008(36MB) 221009(35MB) 221010(35MB) 221011(35MB) 221012(33MB) 221030(35MB) 221031(36MB) 221032(35MB) 221033(36MB)] 2024/11/22-18:41:44.553806 140024802182720 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-18:41:44.553856 140024802182720 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-18:41:44.553885 140024802182720 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-18:41:44.553893 140024802182720 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-18:41:44.553908 140024802182720 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-18:41:45.081166 140024802182720 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221084.sst) /dev/shm/ToplingTemp/Topling-244431-IU74PU 2024/11/22-18:41:46.299056 140024802182720 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221084.sst): first pass time = 1.22's, 110.210'MB/sec 2024/11/22-18:41:48.474480 140024802182720 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221084.sst): old prealloc_size = 147639500, real_size = 33677312 2024/11/22-18:41:48.475157 140024802182720 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221084.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923929 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0317 GB avg-val = 117.27 avg-zval = 34.30 2024/11/22-18:41:48.476024 140024802182720 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221084.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:41:49.826718 140024802182720 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221084.sst): second pass time = 2.17's, 49.826'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.35's, 27.845'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2689.118'MB/sec (index lex order gen) rebuild zvType time = 1.10's, 0.210'MB/sec write SST data time = 0.25's, 150.111'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 36574.567'MB/sec zip my value time = 2.17's, unzip length = 0.108'GB zip my value throughput = 49.826'MB/sec zip pipeline throughput = 49.826'MB/sec entries = 923929 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.27 avg-zval = 34.30 usrkeys = 923929 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.27 avg-zval = 34.30 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923929 bytes = 3118272 } seq expand size = 6467503 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5133 value = 3.4192 dict = 1.00 all = 3.5646 } Zip/UnZip{ index = 0.0951 value = 0.2925 dict = 1.00 all = 0.2805 } ---------------------------- total value len = 0.114815 GB avg = 0.124 KB (by entry num) total key len = 0.025870 GB avg = 0.028 KB total ukey len = 0.018479 GB avg = 0.020 KB total ukey num = 0.000923929 Billion total entry num = 0.000923929 Billion write speed all = 0.003977900 MB/sec (with seq num) write speed all = 0.003768906 MB/sec (without seq num) 2024/11/22-18:41:49.870821 140024802182720 [compaction_job.cc:1948] [default] [JOB 7814] Generated table #221084: 923929 keys, 37652726 bytes, temperature: kWarm 2024/11/22-18:41:49.870941 140024802182720 EVENT_LOG_v1 {"time_micros": 1732272109870865, "cf_name": "default", "job": 7814, "event": "table_file_creation", "file_number": 221084, "file_size": 37652726, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774735, "largest_seqno": 3473439208, "table_properties": {"data_size": 31688496, "index_size": 1757632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25870012, "raw_average_key_size": 28, "raw_value_size": 108347814, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923929, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272105, "oldest_key_time": 0, "file_creation_time": 1732272105, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221084, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:41:49.960926 140024802182720 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221085.sst) /dev/shm/ToplingTemp/Topling-244431-3m7iWU 2024/11/22-18:41:51.629573 140024802182720 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221085.sst): first pass time = 1.67's, 80.437'MB/sec 2024/11/22-18:41:53.361114 140024802182720 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221085.sst): old prealloc_size = 147639500, real_size = 33776656 2024/11/22-18:41:53.361708 140024802182720 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221085.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925095 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0318 GB avg-val = 117.09 avg-zval = 34.36 2024/11/22-18:41:53.362489 140024802182720 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221085.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:41:54.653021 140024802182720 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221085.sst): second pass time = 1.72's, 63.088'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.29's, 29.224'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3133.274'MB/sec (index lex order gen) rebuild zvType time = 1.16's, 0.199'MB/sec write SST data time = 0.13's, 286.142'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 36245.079'MB/sec zip my value time = 1.72's, unzip length = 0.108'GB zip my value throughput = 63.088'MB/sec zip pipeline throughput = 63.088'MB/sec entries = 925095 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.09 avg-zval = 34.36 usrkeys = 925095 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.09 avg-zval = 34.36 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925095 bytes = 3122208 } seq expand size = 6475665 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.5132 value = 3.4077 dict = 1.00 all = 3.5550 } Zip/UnZip{ index = 0.0951 value = 0.2935 dict = 1.00 all = 0.2813 } ---------------------------- total value len = 0.229606 GB avg = 0.124 KB (by entry num) total key len = 0.051773 GB avg = 0.028 KB total ukey len = 0.036980 GB avg = 0.020 KB total ukey num = 0.001849024 Billion total entry num = 0.001849024 Billion write speed all = 0.007954945 MB/sec (with seq num) write speed all = 0.007536751 MB/sec (without seq num) 2024/11/22-18:41:54.706396 140024802182720 [compaction_job.cc:1948] [default] [JOB 7814] Generated table #221085: 925095 keys, 37754216 bytes, temperature: kWarm 2024/11/22-18:41:54.706472 140024802182720 EVENT_LOG_v1 {"time_micros": 1732272114706429, "cf_name": "default", "job": 7814, "event": "table_file_creation", "file_number": 221085, "file_size": 37754216, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3473438342, "table_properties": {"data_size": 31785344, "index_size": 1759872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25902660, "raw_average_key_size": 28, "raw_value_size": 108315146, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925095, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272109, "oldest_key_time": 0, "file_creation_time": 1732272109, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221085, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:41:54.793087 140024802182720 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221086.sst) /dev/shm/ToplingTemp/Topling-244431-nEDkmV 2024/11/22-18:41:56.215290 140024802182720 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221086.sst): first pass time = 1.42's, 94.377'MB/sec 2024/11/22-18:41:57.829185 140024802182720 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221086.sst): old prealloc_size = 147639500, real_size = 33446848 2024/11/22-18:41:57.829744 140024802182720 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221086.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925238 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 117.06 avg-zval = 34.00 2024/11/22-18:41:57.830522 140024802182720 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221086.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:41:59.160751 140024802182720 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221086.sst): second pass time = 1.61's, 67.145'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.33's, 28.107'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3292.484'MB/sec (index lex order gen) rebuild zvType time = 1.08's, 0.215'MB/sec write SST data time = 0.26's, 146.399'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33793.785'MB/sec zip my value time = 1.61's, unzip length = 0.108'GB zip my value throughput = 67.145'MB/sec zip pipeline throughput = 67.145'MB/sec entries = 925238 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.06 avg-zval = 34.00 usrkeys = 925238 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.06 avg-zval = 34.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925238 bytes = 3122688 } seq expand size = 6476666 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5148 value = 3.4433 dict = 1.00 all = 3.5863 } Zip/UnZip{ index = 0.0951 value = 0.2904 dict = 1.00 all = 0.2788 } ---------------------------- total value len = 0.344394 GB avg = 0.124 KB (by entry num) total key len = 0.077679 GB avg = 0.028 KB total ukey len = 0.055485 GB avg = 0.020 KB total ukey num = 0.002774262 Billion total entry num = 0.002774262 Billion write speed all = 0.011931041 MB/sec (with seq num) write speed all = 0.011303665 MB/sec (without seq num) 2024/11/22-18:41:59.202143 140024802182720 [compaction_job.cc:1948] [default] [JOB 7814] Generated table #221086: 925238 keys, 37425008 bytes, temperature: kWarm 2024/11/22-18:41:59.202228 140024802182720 EVENT_LOG_v1 {"time_micros": 1732272119202183, "cf_name": "default", "job": 7814, "event": "table_file_creation", "file_number": 221086, "file_size": 37425008, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774786, "largest_seqno": 3473438883, "table_properties": {"data_size": 31455472, "index_size": 1759872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25906664, "raw_average_key_size": 28, "raw_value_size": 108311144, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925238, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272114, "oldest_key_time": 0, "file_creation_time": 1732272114, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221086, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:41:59.278993 140024802182720 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221087.sst) /dev/shm/ToplingTemp/Topling-244431-r7m0DU 2024/11/22-18:42:00.451380 140024802182720 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221087.sst): first pass time = 1.17's, 114.488'MB/sec 2024/11/22-18:42:01.799827 140024802182720 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221087.sst): old prealloc_size = 147639500, real_size = 34113440 2024/11/22-18:42:01.800399 140024802182720 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221087.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926265 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0321 GB avg-val = 116.90 avg-zval = 34.68 2024/11/22-18:42:01.801031 140024802182720 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221087.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:42:02.916143 140024802182720 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221087.sst): second pass time = 1.35's, 80.369'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.12's, 33.409'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3211.209'MB/sec (index lex order gen) rebuild zvType time = 0.88's, 0.264'MB/sec write SST data time = 0.24's, 156.180'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31424.308'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.369'MB/sec zip pipeline throughput = 80.369'MB/sec entries = 926265 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.90 avg-zval = 34.68 usrkeys = 926265 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.78 avg-zval = 34.68 TagRS{ kinds = RS_Key0_TagN bytes = 144840 } TagArray{ size = 645137 bytes = 2177360 } seq expand size = 4515959 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0321 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5143 value = 3.3712 dict = 1.00 all = 3.5989 } Zip/UnZip{ index = 0.0951 value = 0.2966 dict = 1.00 all = 0.2779 } ---------------------------- total value len = 0.457192 GB avg = 0.124 KB (by entry num) total key len = 0.103615 GB avg = 0.028 KB total ukey len = 0.074011 GB avg = 0.020 KB total ukey num = 0.003700527 Billion total entry num = 0.003700527 Billion write speed all = 0.015851043 MB/sec (with seq num) write speed all = 0.015014289 MB/sec (without seq num) 2024/11/22-18:42:02.954876 140024802182720 [compaction_job.cc:1948] [default] [JOB 7814] Generated table #221087: 926265 keys, 37293634 bytes, temperature: kWarm 2024/11/22-18:42:02.954964 140024802182720 EVENT_LOG_v1 {"time_micros": 1732272122954916, "cf_name": "default", "job": 7814, "event": "table_file_creation", "file_number": 221087, "file_size": 37293634, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3473439113, "table_properties": {"data_size": 32119760, "index_size": 1761920, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25935420, "raw_average_key_size": 28, "raw_value_size": 108282410, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926265, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272119, "oldest_key_time": 0, "file_creation_time": 1732272119, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221087, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:42:03.003574 140024802182720 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221088.sst) /dev/shm/ToplingTemp/Topling-244431-3A3ATU 2024/11/22-18:42:04.075830 140024802182720 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221088.sst): first pass time = 1.07's, 125.180'MB/sec 2024/11/22-18:42:05.509146 140024802182720 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221088.sst): old prealloc_size = 147639500, real_size = 34972656 2024/11/22-18:42:05.509834 140024802182720 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221088.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1081591 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1039 GB zip-val = 0.0326 GB avg-val = 96.09 avg-zval = 30.18 2024/11/22-18:42:05.509886 140024802182720 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221088.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:42:06.135839 140024802182720 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221088.sst): second pass time = 1.43's, 72.547'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.63's, 57.044'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3088.255'MB/sec (index lex order gen) rebuild zvType time = 0.49's, 0.557'MB/sec write SST data time = 0.14's, 254.934'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.034'MB, throughput = 29163.828'MB/sec zip my value time = 1.43's, unzip length = 0.104'GB zip my value throughput = 72.547'MB/sec zip pipeline throughput = 72.547'MB/sec entries = 1081591 avg-key = 28.00 avg-zkey = 1.90 avg-val = 96.09 avg-zval = 30.18 usrkeys = 1081591 avg-key = 20.00 avg-zkey = 1.90 avg-val = 96.09 avg-zval = 30.18 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0216 GB value = 0.1039 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0326 GB dict = 1.03 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5164 value = 3.1837 dict = 1.00 all = 3.7546 } Zip/UnZip{ index = 0.0951 value = 0.3141 dict = 1.00 all = 0.2663 } ---------------------------- total value len = 0.561126 GB avg = 0.117 KB (by entry num) total key len = 0.133899 GB avg = 0.028 KB total ukey len = 0.095642 GB avg = 0.020 KB total ukey num = 0.004782118 Billion total entry num = 0.004782118 Billion write speed all = 0.019642881 MB/sec (with seq num) write speed all = 0.018561659 MB/sec (without seq num) 2024/11/22-18:42:06.169815 140024802182720 [compaction_job.cc:1948] [default] [JOB 7814] Generated table #221088: 1081591 keys, 35747420 bytes, temperature: kWarm 2024/11/22-18:42:06.169875 140024802182720 EVENT_LOG_v1 {"time_micros": 1732272126169845, "cf_name": "default", "job": 7814, "event": "table_file_creation", "file_number": 221088, "file_size": 35747420, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 32645088, "index_size": 2056960, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30284548, "raw_average_key_size": 28, "raw_value_size": 103933243, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1081591, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272122, "oldest_key_time": 0, "file_creation_time": 1732272122, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221088, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:42:06.206652 140024802182720 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221089.sst) /dev/shm/ToplingTemp/Topling-244431-Hg7hoW 2024/11/22-18:42:07.021815 140024802182720 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221089.sst): first pass time = 0.82's, 164.662'MB/sec 2024/11/22-18:42:08.362276 140024802182720 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221089.sst): old prealloc_size = 147639500, real_size = 33778896 2024/11/22-18:42:08.362849 140024802182720 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221089.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922934 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0318 GB avg-val = 117.43 avg-zval = 34.43 2024/11/22-18:42:08.363660 140024802182720 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221089.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:42:09.633252 140024802182720 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221089.sst): second pass time = 1.34's, 80.920'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.27's, 29.840'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3207.224'MB/sec (index lex order gen) rebuild zvType time = 1.02's, 0.225'MB/sec write SST data time = 0.25's, 153.625'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32058.261'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.920'MB/sec zip pipeline throughput = 80.920'MB/sec entries = 922934 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.43 avg-zval = 34.43 usrkeys = 922934 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.23 avg-zval = 34.43 TagRS{ kinds = RS_Key0_TagN bytes = 144320 } TagArray{ size = 897250 bytes = 3140384 } seq expand size = 6280750 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.4438 value = 3.4101 dict = 1.00 all = 3.5390 } Zip/UnZip{ index = 0.0958 value = 0.2932 dict = 1.00 all = 0.2826 } ---------------------------- total value len = 0.675782 GB avg = 0.118 KB (by entry num) total key len = 0.159741 GB avg = 0.028 KB total ukey len = 0.114101 GB avg = 0.020 KB total ukey num = 0.005705052 Billion total entry num = 0.005705052 Billion write speed all = 0.023611336 MB/sec (with seq num) write speed all = 0.022321569 MB/sec (without seq num) 2024/11/22-18:42:09.676904 140024802182720 [compaction_job.cc:1948] [default] [JOB 7814] Generated table #221089: 922934 keys, 37925216 bytes, temperature: kWarm 2024/11/22-18:42:09.676989 140024802182720 EVENT_LOG_v1 {"time_micros": 1732272129676944, "cf_name": "default", "job": 7814, "event": "table_file_creation", "file_number": 221089, "file_size": 37925216, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3473438058, "table_properties": {"data_size": 31780544, "index_size": 1767424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25842152, "raw_average_key_size": 28, "raw_value_size": 108375680, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922934, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272126, "oldest_key_time": 0, "file_creation_time": 1732272126, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221089, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:42:09.749657 140024802182720 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221090.sst) /dev/shm/ToplingTemp/Topling-244431-c5xW1T 2024/11/22-18:42:10.839704 140024802182720 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221090.sst): first pass time = 1.09's, 123.137'MB/sec 2024/11/22-18:42:12.280739 140024802182720 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221090.sst): old prealloc_size = 147639500, real_size = 32873408 2024/11/22-18:42:12.281328 140024802182720 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221090.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926000 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 116.94 avg-zval = 33.35 2024/11/22-18:42:12.282097 140024802182720 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221090.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:42:13.207014 140024802182720 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221090.sst): second pass time = 1.44's, 75.180'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 39.788'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3109.190'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.291'MB/sec write SST data time = 0.13's, 285.722'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 30860.374'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.180'MB/sec zip pipeline throughput = 82.171'MB/sec entries = 926000 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.94 avg-zval = 33.35 usrkeys = 926000 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.94 avg-zval = 33.35 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926000 bytes = 3125264 } seq expand size = 6482000 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5128 value = 3.5068 dict = 1.00 all = 3.6419 } Zip/UnZip{ index = 0.0951 value = 0.2852 dict = 1.00 all = 0.2746 } ---------------------------- total value len = 0.790554 GB avg = 0.119 KB (by entry num) total key len = 0.185669 GB avg = 0.028 KB total ukey len = 0.132621 GB avg = 0.020 KB total ukey num = 0.006631052 Billion total entry num = 0.006631052 Billion write speed all = 0.027584632 MB/sec (with seq num) write speed all = 0.026085670 MB/sec (without seq num) 2024/11/22-18:42:13.240294 140024802182720 [compaction_job.cc:1948] [default] [JOB 7814] Generated table #221090: 926000 keys, 36853584 bytes, temperature: kWarm 2024/11/22-18:42:13.240358 140024802182720 EVENT_LOG_v1 {"time_micros": 1732272133240323, "cf_name": "default", "job": 7814, "event": "table_file_creation", "file_number": 221090, "file_size": 36853584, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3365258144, "largest_seqno": 3473437921, "table_properties": {"data_size": 30880048, "index_size": 1761664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25928000, "raw_average_key_size": 28, "raw_value_size": 108289736, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926000, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272129, "oldest_key_time": 0, "file_creation_time": 1732272129, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221090, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:42:13.275967 140024802182720 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221091.sst) /dev/shm/ToplingTemp/Topling-244431-dACmJX 2024/11/22-18:42:14.185838 140024802182720 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221091.sst): first pass time = 0.91's, 147.520'MB/sec 2024/11/22-18:42:15.492711 140024802182720 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221091.sst): old prealloc_size = 147639500, real_size = 31494560 2024/11/22-18:42:15.493272 140024802182720 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221091.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919538 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 117.96 avg-zval = 32.10 2024/11/22-18:42:15.494033 140024802182720 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221091.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:42:16.355577 140024802182720 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221091.sst): second pass time = 1.31's, 83.044'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 41.228'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3260.380'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.313'MB/sec write SST data time = 0.13's, 277.047'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 31139.454'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 83.044'MB/sec zip pipeline throughput = 83.044'MB/sec entries = 919538 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.96 avg-zval = 32.10 usrkeys = 919538 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.96 avg-zval = 32.10 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919538 bytes = 3218400 } seq expand size = 6436766 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0295 GB dict = 1.08 MB all = 0.0356 GB } UnZip/Zip{ index = 10.5108 value = 3.6751 dict = 1.00 all = 3.7729 } Zip/UnZip{ index = 0.0951 value = 0.2721 dict = 1.00 all = 0.2650 } ---------------------------- total value len = 0.905461 GB avg = 0.120 KB (by entry num) total key len = 0.211417 GB avg = 0.028 KB total ukey len = 0.151012 GB avg = 0.020 KB total ukey num = 0.007550590 Billion total entry num = 0.007550590 Billion write speed all = 0.031556227 MB/sec (with seq num) write speed all = 0.029849554 MB/sec (without seq num) 2024/11/22-18:42:16.389342 140024802182720 [compaction_job.cc:1948] [default] [JOB 7814] Generated table #221091: 919538 keys, 35573776 bytes, temperature: kWarm 2024/11/22-18:42:16.389406 140024802182720 EVENT_LOG_v1 {"time_micros": 1732272136389372, "cf_name": "default", "job": 7814, "event": "table_file_creation", "file_number": 221091, "file_size": 35573776, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3473437795, "table_properties": {"data_size": 29514768, "index_size": 1749696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25747064, "raw_average_key_size": 28, "raw_value_size": 108470712, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 919538, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272133, "oldest_key_time": 0, "file_creation_time": 1732272133, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221091, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:42:16.426798 140024802182720 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221092.sst) /dev/shm/ToplingTemp/Topling-244431-txsAEU 2024/11/22-18:42:17.251000 140024802182720 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221092.sst): first pass time = 0.82's, 162.854'MB/sec 2024/11/22-18:42:18.601296 140024802182720 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221092.sst): old prealloc_size = 147639500, real_size = 33367440 2024/11/22-18:42:18.601843 140024802182720 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221092.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924497 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 117.18 avg-zval = 33.93 2024/11/22-18:42:18.602608 140024802182720 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221092.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:42:19.533584 140024802182720 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221092.sst): second pass time = 1.35's, 80.271'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.226'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3355.119'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.288'MB/sec write SST data time = 0.13's, 291.077'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 29653.051'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.271'MB/sec zip pipeline throughput = 80.271'MB/sec entries = 924497 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.18 avg-zval = 33.93 usrkeys = 924497 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.18 avg-zval = 33.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924497 bytes = 3235760 } seq expand size = 6471479 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4430 value = 3.4538 dict = 1.00 all = 3.5830 } Zip/UnZip{ index = 0.0958 value = 0.2895 dict = 1.00 all = 0.2791 } ---------------------------- total value len = 1.020265 GB avg = 0.120 KB (by entry num) total key len = 0.237302 GB avg = 0.028 KB total ukey len = 0.169502 GB avg = 0.020 KB total ukey num = 0.008475087 Billion total entry num = 0.008475087 Billion write speed all = 0.035528069 MB/sec (with seq num) write speed all = 0.033612602 MB/sec (without seq num) 2024/11/22-18:42:19.566265 140024802182720 [compaction_job.cc:1948] [default] [JOB 7814] Generated table #221092: 924497 keys, 37459556 bytes, temperature: kWarm 2024/11/22-18:42:19.566334 140024802182720 EVENT_LOG_v1 {"time_micros": 1732272139566297, "cf_name": "default", "job": 7814, "event": "table_file_creation", "file_number": 221092, "file_size": 37459556, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701122, "largest_seqno": 3473436673, "table_properties": {"data_size": 31365568, "index_size": 1770560, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25885916, "raw_average_key_size": 28, "raw_value_size": 108331930, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924497, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272136, "oldest_key_time": 0, "file_creation_time": 1732272136, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221092, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:42:19.605043 140024802182720 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221093.sst) /dev/shm/ToplingTemp/Topling-244431-2lWQmW 2024/11/22-18:42:20.452560 140024802182720 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221093.sst): first pass time = 0.85's, 158.374'MB/sec 2024/11/22-18:42:21.788962 140024802182720 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221093.sst): old prealloc_size = 147639500, real_size = 33557072 2024/11/22-18:42:21.789507 140024802182720 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221093.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924547 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 117.17 avg-zval = 34.14 2024/11/22-18:42:21.790314 140024802182720 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221093.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:42:22.692054 140024802182720 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221093.sst): second pass time = 1.34's, 81.101'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 41.690'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3371.805'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.299'MB/sec write SST data time = 0.13's, 291.456'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32739.646'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.101'MB/sec zip pipeline throughput = 81.101'MB/sec entries = 924547 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.17 avg-zval = 34.14 usrkeys = 924547 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.17 avg-zval = 34.14 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924547 bytes = 3235936 } seq expand size = 6471829 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5131 value = 3.4318 dict = 1.00 all = 3.5650 } Zip/UnZip{ index = 0.0951 value = 0.2914 dict = 1.00 all = 0.2805 } ---------------------------- total value len = 1.135067 GB avg = 0.121 KB (by entry num) total key len = 0.263190 GB avg = 0.028 KB total ukey len = 0.187993 GB avg = 0.020 KB total ukey num = 0.009399634 Billion total entry num = 0.009399634 Billion write speed all = 0.039499228 MB/sec (with seq num) write speed all = 0.037374993 MB/sec (without seq num) 2024/11/22-18:42:22.726717 140024802182720 [compaction_job.cc:1948] [default] [JOB 7814] Generated table #221093: 924547 keys, 37649152 bytes, temperature: kWarm 2024/11/22-18:42:22.726788 140024802182720 EVENT_LOG_v1 {"time_micros": 1732272142726749, "cf_name": "default", "job": 7814, "event": "table_file_creation", "file_number": 221093, "file_size": 37649152, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3326330223, "largest_seqno": 3473439340, "table_properties": {"data_size": 31566912, "index_size": 1758848, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25887316, "raw_average_key_size": 28, "raw_value_size": 108330530, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924547, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272139, "oldest_key_time": 0, "file_creation_time": 1732272139, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221093, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:42:22.765714 140024802182720 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221094.sst) /dev/shm/ToplingTemp/Topling-244431-WsRNaV 2024/11/22-18:42:23.682702 140024802182720 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221094.sst): first pass time = 0.92's, 146.375'MB/sec 2024/11/22-18:42:25.002702 140024802182720 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221094.sst): old prealloc_size = 147639500, real_size = 33116912 2024/11/22-18:42:25.003263 140024802182720 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221094.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922754 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0311 GB avg-val = 117.45 avg-zval = 33.74 2024/11/22-18:42:25.003852 140024802182720 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221094.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:42:25.902542 140024802182720 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221094.sst): second pass time = 1.32's, 82.146'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 40.261'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3246.041'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.300'MB/sec write SST data time = 0.13's, 278.306'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32742.208'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.146'MB/sec zip pipeline throughput = 82.146'MB/sec entries = 922754 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.45 avg-zval = 33.74 usrkeys = 922754 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.02 avg-zval = 33.74 TagRS{ kinds = RS_Key0_TagN bytes = 144280 } TagArray{ size = 602481 bytes = 2108704 } seq expand size = 4217367 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.5115 value = 3.4815 dict = 1.00 all = 3.7048 } Zip/UnZip{ index = 0.0951 value = 0.2872 dict = 1.00 all = 0.2699 } ---------------------------- total value len = 1.247665 GB avg = 0.121 KB (by entry num) total key len = 0.289027 GB avg = 0.028 KB total ukey len = 0.206448 GB avg = 0.020 KB total ukey num = 0.010322388 Billion total entry num = 0.010322388 Billion write speed all = 0.043405935 MB/sec (with seq num) write speed all = 0.041073377 MB/sec (without seq num) 2024/11/22-18:42:25.940764 140024802182720 [compaction_job.cc:1948] [default] [JOB 7814] Generated table #221094: 922754 keys, 36228118 bytes, temperature: kWarm 2024/11/22-18:42:25.940836 140024802182720 EVENT_LOG_v1 {"time_micros": 1732272145940803, "cf_name": "default", "job": 7814, "event": "table_file_creation", "file_number": 221094, "file_size": 36228118, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3473439079, "table_properties": {"data_size": 31130336, "index_size": 1755712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25837112, "raw_average_key_size": 28, "raw_value_size": 108380712, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922754, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272142, "oldest_key_time": 0, "file_creation_time": 1732272142, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221094, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:42:25.978131 140024802182720 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221095.sst) /dev/shm/ToplingTemp/Topling-244431-KiiT8T 2024/11/22-18:42:26.914587 140024802182720 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221095.sst): first pass time = 0.94's, 143.331'MB/sec 2024/11/22-18:42:28.220029 140024802182720 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221095.sst): old prealloc_size = 147639500, real_size = 33342704 2024/11/22-18:42:28.220616 140024802182720 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221095.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925842 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.97 avg-zval = 33.86 2024/11/22-18:42:28.221395 140024802182720 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221095.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:42:29.164190 140024802182720 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221095.sst): second pass time = 1.30's, 82.996'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.531'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3182.969'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.284'MB/sec write SST data time = 0.13's, 289.412'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35132.167'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 82.996'MB/sec zip pipeline throughput = 82.996'MB/sec entries = 925842 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.97 avg-zval = 33.86 usrkeys = 925842 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.97 avg-zval = 33.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925842 bytes = 3124736 } seq expand size = 6480894 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5133 value = 3.4544 dict = 1.00 all = 3.5961 } Zip/UnZip{ index = 0.0951 value = 0.2895 dict = 1.00 all = 0.2781 } ---------------------------- total value len = 1.362440 GB avg = 0.121 KB (by entry num) total key len = 0.314950 GB avg = 0.028 KB total ukey len = 0.224965 GB avg = 0.020 KB total ukey num = 0.011248230 Billion total entry num = 0.011248230 Billion write speed all = 0.047375792 MB/sec (with seq num) write speed all = 0.044834255 MB/sec (without seq num) 2024/11/22-18:42:29.201221 140024802182720 [compaction_job.cc:1948] [default] [JOB 7814] Generated table #221095: 925842 keys, 37322678 bytes, temperature: kWarm 2024/11/22-18:42:29.201288 140024802182720 EVENT_LOG_v1 {"time_micros": 1732272149201254, "cf_name": "default", "job": 7814, "event": "table_file_creation", "file_number": 221095, "file_size": 37322678, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3388810268, "largest_seqno": 3473439247, "table_properties": {"data_size": 31349792, "index_size": 1761280, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25923576, "raw_average_key_size": 28, "raw_value_size": 108294156, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925842, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272145, "oldest_key_time": 0, "file_creation_time": 1732272145, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221095, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:42:29.237819 140024802182720 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221096.sst) /dev/shm/ToplingTemp/Topling-244431-SV8tOU 2024/11/22-18:42:29.389028 140024802182720 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221096.sst): first pass time = 0.15's, 237.512'MB/sec 2024/11/22-18:42:29.725019 140024802182720 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221096.sst): old prealloc_size = 147639500, real_size = 10028848 2024/11/22-18:42:29.725226 140024802182720 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221096.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 246209 prefix = 4 raw-key = 0.0039 GB zip-key = 0.0005 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0290 GB zip-val = 0.0095 GB avg-val = 117.82 avg-zval = 38.57 2024/11/22-18:42:29.725464 140024802182720 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221096.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0006 GB, waited 0.000 sec, Key+Value = 0.036 GB 2024/11/22-18:42:29.966738 140024802182720 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00/221096.sst): second pass time = 0.34's, 86.494'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.24's, 45.916'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2787.447'MB/sec (index lex order gen) rebuild zvType time = 0.21's, 0.298'MB/sec write SST data time = 0.04's, 314.649'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.297'MB, throughput = 9587.300'MB/sec zip my value time = 0.34's, unzip length = 0.029'GB zip my value throughput = 86.494'MB/sec zip pipeline throughput = 86.494'MB/sec entries = 246209 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.82 avg-zval = 38.57 usrkeys = 246209 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.82 avg-zval = 38.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 246209 bytes = 830976 } seq expand size = 1723463 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0049 GB value = 0.0290 GB dict = 0.30 MB all = 0.0359 GB } __ZipSize{ index = 0.0005 GB value = 0.0095 GB dict = 0.30 MB all = 0.0111 GB } UnZip/Zip{ index = 10.4838 value = 3.0544 dict = 1.00 all = 3.2351 } Zip/UnZip{ index = 0.0954 value = 0.3274 dict = 1.00 all = 0.3091 } ---------------------------- total value len = 1.393173 GB avg = 0.121 KB (by entry num) total key len = 0.321844 GB avg = 0.028 KB total ukey len = 0.229889 GB avg = 0.020 KB total ukey num = 0.011494439 Billion total entry num = 0.011494439 Billion write speed all = 0.048437406 MB/sec (with seq num) write speed all = 0.045840296 MB/sec (without seq num) 2024/11/22-18:42:29.980972 140024802182720 [compaction_job.cc:1948] [default] [JOB 7814] Generated table #221096: 246209 keys, 11097903 bytes, temperature: kWarm 2024/11/22-18:42:29.981037 140024802182720 EVENT_LOG_v1 {"time_micros": 1732272149981000, "cf_name": "default", "job": 7814, "event": "table_file_creation", "file_number": 221096, "file_size": 11097903, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 3472470721, "table_properties": {"data_size": 9497440, "index_size": 469696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 6893852, "raw_average_key_size": 28, "raw_value_size": 29009106, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 246209, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272149, "oldest_key_time": 0, "file_creation_time": 1732272149, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221096, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:42:29.994540 140024802182720 [compaction_job.cc:703] job-07814: subcompact[0], size: 0.455983 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07814/att-00 [221084,221085,221086,221087,221088,221089,221090,221091,221092,221093,221094,221095,221096] 2024/11/22-18:42:30.075880 140024802182720 (Original Log Time 2024/11/22-18:41:44.544545) EVENT_LOG_v1 {"time_micros": 1732272104544484, "job": 7814, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [221076], "files_L3": [221028, 221029, 221006, 221007, 221008, 221009, 221010, 221011, 221012, 221030, 221031, 221032, 221033], "score": 0.9541, "input_data_size": 482693909, "oldest_snapshot_seqno": 3473491288} 2024/11/22-18:42:30.075883 140024802182720 (Original Log Time 2024/11/22-18:42:30.075204) [compaction_job.cc:2022] [default] [JOB 7814] Compacted 1@2 + 13@3 files to L3 => 455982987 bytes 2024/11/22-18:42:30.075885 140024802182720 (Original Log Time 2024/11/22-18:42:30.075815) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 13 0 0 0] max score 0.02, MB/sec: 10.6 rd, 10.0 wr, level 3, files in(1, 13) out(13 +0 blob) MB in(27.1, 433.2 +0.0 blob) out(434.9 +0.0 blob), read-write-amplify(33.0) write-amplify(16.1) OK, records in: 11605114, records dropped: 110675 output_compression: Snappy 2024/11/22-18:42:30.075891 140024802182720 (Original Log Time 2024/11/22-18:42:30.075860) EVENT_LOG_v1 {"time_micros": 1732272150075839, "cf": "default", "job": 7814, "event": "compaction_finished", "compaction_time_micros": 45450003, "compaction_time_cpu_micros": 20779530, "output_level": 3, "num_output_files": 13, "total_output_size": 455982987, "num_input_records": 11605114, "num_output_records": 11494439, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 324866918, "file_range_sync_nanos": 0, "file_fsync_nanos": 13343, "file_prepare_write_nanos": 33081, "lsm_state": [0, 0, 0, 13, 0, 0, 0]} 2024/11/22-18:42:30.198663 140024802182720 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732272104,"executesMs":45655,"compactionJobId":7814,"attempt":0,"compactionInputRawBytes":1662232461,"compactionInputZipBytes":482693909,"compactionOutputRawBytes":0,"compactionOutputZipBytes":455982987,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-9270f7fb24e863eb0e9be38255ea36a7-88b66bdd5cbf7830-00"}}, response = 2024/11/22-18:42:30.198741 140024802182720 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-07814/att-00: olev 3, work 45.656 s, result 5.030 ms, install 0.709 ms, input{raw 1.548 GiB zip 460.333 MiB}