2024/11/20-16:18:09.941601 140257248339520 DEBG dcompact_worker.cpp:2219: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00: fork to child time = 0.025358 sec 2024/11/20-16:18:09.942365 140257248339520 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00 2024/11/20-16:18:09.950059 140257248339520 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00 2024/11/20-16:18:09.952352 140257248339520 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/20-16:18:09.952676 140257248339520 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/20-16:18:10.000958 140257248339520 [version_set.cc:6211] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T12.20.00/.rocksdb/job-15320/att-00/MANIFEST-019309 2024/11/20-16:18:10.058871 140257248339520 [version_set.cc:6254] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T12.20.00/.rocksdb/job-15320/att-00/MANIFEST-019309 succeeded,manifest_file_number is 19309, next_file_number is 44536, last_sequence is 764392853, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 44323 2024/11/20-16:18:10.058881 140257248339520 [version_set.cc:6269] Column family [default] (ID 0), log number is 0 2024/11/20-16:18:10.059027 140257248339520 [version_set.cc:5724] Creating manifest 44536 2024/11/20-16:18:10.088725 140257248339520 DEBG dcompact_worker.cpp:1225: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/20-16:18:10.110423 140257248339520 [compaction_job.cc:2437] [default] [JOB 15320] Compacting 1@2 + 17@3 files to L3, score 0.95, subcompactions 1 : 1 2024/11/20-16:18:10.110437 140257248339520 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 11958 Base level 2, inputs: [44496(54MB)], [44488(37MB) 44489(6057KB) 44422(37MB) 44423(37MB) 44424(36MB) 44425(36MB) 44426(38MB) 44427(35MB) 44428(35MB) 44429(37MB) 44430(37MB) 44431(17MB) 44316(38MB) 44317(36MB) 44318(36MB) 44319(37MB) 44526(38MB)] 2024/11/20-16:18:10.121751 140257248339520 [top_zip_table.cc:59] core git_version_hash_info_is:commit 81964c21ebbcc178d8e99dae0fbf61bded2729cc Author: rockeet Date: Mon Nov 11 21:28:46 2024 +0800 Makefile: fix gen git-version-*.cpp GIT_PATH_ARG = :!src/terark/fsa :!tools/fsa :!src/terark/zbs :!tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-16:18:10.121775 140257248339520 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit 5dfecb7d107196f044ff5bf78dc39d2e725b9d9b Author: leipeng Date: Thu Nov 7 15:22:45 2024 +0800 Makefile: fix for x86_64 CPU older than haswell GIT_PATH_ARG = src/terark/fsa tools/fsa g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-16:18:10.121794 140257248339520 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit d44dd5ff72707128fe3c916077b43d2093c01f7c Author: leipeng Date: Sun Oct 27 23:38:26 2024 +0800 Fix for msvc, when toplingdb on msvc GIT_PATH_ARG = src/terark/zbs tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-16:18:10.121803 140257248339520 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit 79614a7e7dc27362fc5151b0797eb7d83c674c7b Author: rockeet Date: Sun Nov 10 15:02:21 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-16:18:10.121813 140257248339520 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit c46b65a52d9c74aa8b87aeb8438620ff2dbc669c Author: rockeet Date: Sun Nov 10 15:02:42 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-16:18:10.910923 140257248339520 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044537.sst) /dev/shm/ToplingTemp/Topling-561823-OCsQPA 2024/11/20-16:18:11.506768 140257248339520 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044537.sst): first pass time = 0.60's, 225.273'MB/sec 2024/11/20-16:18:12.667596 140257248339520 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044537.sst): old prealloc_size = 147639500, real_size = 35530144 2024/11/20-16:18:12.668233 140257248339520 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044537.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927863 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0335 GB avg-val = 116.65 avg-zval = 36.14 2024/11/20-16:18:12.669065 140257248339520 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044537.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-16:18:13.493513 140257248339520 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044537.sst): second pass time = 1.16's, 93.315'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.984'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2937.897'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.12's, 320.908'MB/sec dict compress time = 4051619406.19's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 19013.910'MB/sec zip my value time = 1.16's, unzip length = 0.108'GB zip my value throughput = 93.315'MB/sec zip pipeline throughput = 93.315'MB/sec entries = 927863 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.65 avg-zval = 36.14 usrkeys = 927863 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.65 avg-zval = 36.14 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927863 bytes = 3247536 } seq expand size = 6495041 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0335 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.5103 value = 3.2279 dict = 1.00 all = 3.3867 } Zip/UnZip{ index = 0.0951 value = 0.3098 dict = 1.00 all = 0.2953 } ---------------------------- total value len = 0.114733 GB avg = 0.124 KB (by entry num) total key len = 0.025980 GB avg = 0.028 KB total ukey len = 0.018557 GB avg = 0.020 KB total ukey num = 0.000927863 Billion total entry num = 0.000927863 Billion write speed all = 0.005205484 MB/sec (with seq num) write speed all = 0.004930884 MB/sec (without seq num) 2024/11/20-16:18:13.540748 140257248339520 [compaction_job.cc:1948] [default] [JOB 15320] Generated table #44537: 927863 keys, 39631380 bytes, temperature: kWarm 2024/11/20-16:18:13.540830 140257248339520 EVENT_LOG_v1 {"time_micros": 1732090693540791, "cf_name": "default", "job": 15320, "event": "table_file_creation", "file_number": 44537, "file_size": 39631380, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 542657474, "largest_seqno": 763673341, "table_properties": {"data_size": 33532368, "index_size": 1765632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25980164, "raw_average_key_size": 28, "raw_value_size": 108237650, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927863, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732090690, "oldest_key_time": 0, "file_creation_time": 1732090690, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 44537, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-16:18:13.575414 140257248339520 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044538.sst) /dev/shm/ToplingTemp/Topling-561823-SnIHiA 2024/11/20-16:18:14.440075 140257248339520 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044538.sst): first pass time = 0.86's, 155.232'MB/sec 2024/11/20-16:18:15.587374 140257248339520 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044538.sst): old prealloc_size = 147639500, real_size = 35213600 2024/11/20-16:18:15.587947 140257248339520 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044538.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924903 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1083 GB zip-val = 0.0332 GB avg-val = 117.12 avg-zval = 35.88 2024/11/20-16:18:15.588743 140257248339520 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044538.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-16:18:16.388599 140257248339520 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044538.sst): second pass time = 1.15's, 94.478'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.80's, 48.850'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3268.881'MB/sec (index lex order gen) rebuild zvType time = 0.67's, 0.344'MB/sec write SST data time = 0.13's, 304.557'MB/sec dict compress time = 0.00's, 1080942.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33572.755'MB/sec zip my value time = 1.15's, unzip length = 0.108'GB zip my value throughput = 94.478'MB/sec zip pipeline throughput = 94.478'MB/sec entries = 924903 avg-key = 28.00 avg-zkey = 1.94 avg-val = 117.12 avg-zval = 35.88 usrkeys = 924903 avg-key = 20.00 avg-zkey = 1.94 avg-val = 123.22 avg-zval = 35.88 TagRS{ kinds = RS_Key0_TagN bytes = 144600 } TagArray{ size = 805941 bytes = 2921552 } seq expand size = 5641587 multi value expand size = 0 cnt WriteAppend = 36 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0391 GB } UnZip/Zip{ index = 10.3079 value = 3.2639 dict = 1.00 all = 3.4293 } Zip/UnZip{ index = 0.0970 value = 0.3064 dict = 1.00 all = 0.2916 } ---------------------------- total value len = 0.228695 GB avg = 0.123 KB (by entry num) total key len = 0.051877 GB avg = 0.028 KB total ukey len = 0.037055 GB avg = 0.020 KB total ukey num = 0.001852766 Billion total entry num = 0.001852766 Billion write speed all = 0.010378286 MB/sec (with seq num) write speed all = 0.009830019 MB/sec (without seq num) 2024/11/20-16:18:16.435303 140257248339520 [compaction_job.cc:1948] [default] [JOB 15320] Generated table #44538: 924903 keys, 39139006 bytes, temperature: kWarm 2024/11/20-16:18:16.435363 140257248339520 EVENT_LOG_v1 {"time_micros": 1732090696435334, "cf_name": "default", "job": 15320, "event": "table_file_creation", "file_number": 44538, "file_size": 39139006, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 763672086, "table_properties": {"data_size": 33187600, "index_size": 1794560, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25897284, "raw_average_key_size": 28, "raw_value_size": 108320554, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924903, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732090693, "oldest_key_time": 0, "file_creation_time": 1732090693, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 44538, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-16:18:16.470813 140257248339520 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044539.sst) /dev/shm/ToplingTemp/Topling-561823-tPh1vA 2024/11/20-16:18:17.286251 140257248339520 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044539.sst): first pass time = 0.82's, 164.603'MB/sec 2024/11/20-16:18:18.402647 140257248339520 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044539.sst): old prealloc_size = 147639500, real_size = 34514480 2024/11/20-16:18:18.403232 140257248339520 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044539.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925014 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0325 GB avg-val = 117.10 avg-zval = 35.16 2024/11/20-16:18:18.404020 140257248339520 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044539.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-16:18:19.242941 140257248339520 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044539.sst): second pass time = 1.12's, 97.097'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 45.809'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3131.684'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.324'MB/sec write SST data time = 0.13's, 306.106'MB/sec dict compress time = 0.00's, 1080958.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 30029.948'MB/sec zip my value time = 1.12's, unzip length = 0.108'GB zip my value throughput = 97.097'MB/sec zip pipeline throughput = 97.097'MB/sec entries = 925014 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.10 avg-zval = 35.16 usrkeys = 925014 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.10 avg-zval = 35.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925014 bytes = 3121936 } seq expand size = 6475098 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.5123 value = 3.3305 dict = 1.00 all = 3.4869 } Zip/UnZip{ index = 0.0951 value = 0.3003 dict = 1.00 all = 0.2868 } ---------------------------- total value len = 0.343487 GB avg = 0.124 KB (by entry num) total key len = 0.077778 GB avg = 0.028 KB total ukey len = 0.055556 GB avg = 0.020 KB total ukey num = 0.002777780 Billion total entry num = 0.002777780 Billion write speed all = 0.015580830 MB/sec (with seq num) write speed all = 0.014758923 MB/sec (without seq num) 2024/11/20-16:18:19.289856 140257248339520 [compaction_job.cc:1948] [default] [JOB 15320] Generated table #44539: 925014 keys, 38491982 bytes, temperature: kWarm 2024/11/20-16:18:19.289925 140257248339520 EVENT_LOG_v1 {"time_micros": 1732090699289888, "cf_name": "default", "job": 15320, "event": "table_file_creation", "file_number": 44539, "file_size": 38491982, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 635932289, "largest_seqno": 763672556, "table_properties": {"data_size": 32523168, "index_size": 1759872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25900392, "raw_average_key_size": 28, "raw_value_size": 108317444, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925014, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732090696, "oldest_key_time": 0, "file_creation_time": 1732090696, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 44539, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-16:18:19.323802 140257248339520 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044540.sst) /dev/shm/ToplingTemp/Topling-561823-edhlLB 2024/11/20-16:18:20.131027 140257248339520 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044540.sst): first pass time = 0.81's, 166.277'MB/sec 2024/11/20-16:18:21.302602 140257248339520 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044540.sst): old prealloc_size = 147639500, real_size = 34813504 2024/11/20-16:18:21.303205 140257248339520 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044540.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923681 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0328 GB avg-val = 117.31 avg-zval = 35.54 2024/11/20-16:18:21.303968 140257248339520 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044540.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-16:18:22.139353 140257248339520 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044540.sst): second pass time = 1.17's, 92.557'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.356'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3089.433'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.12's, 310.873'MB/sec dict compress time = 0.00's, 1082428.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 29132.768'MB/sec zip my value time = 1.17's, unzip length = 0.108'GB zip my value throughput = 92.557'MB/sec zip pipeline throughput = 92.557'MB/sec entries = 923681 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.31 avg-zval = 35.54 usrkeys = 923681 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.31 avg-zval = 35.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923681 bytes = 3117440 } seq expand size = 6465767 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0328 GB dict = 1.08 MB all = 0.0388 GB } UnZip/Zip{ index = 10.5128 value = 3.3010 dict = 1.00 all = 3.4603 } Zip/UnZip{ index = 0.0951 value = 0.3029 dict = 1.00 all = 0.2890 } ---------------------------- total value len = 0.458308 GB avg = 0.124 KB (by entry num) total key len = 0.103641 GB avg = 0.028 KB total ukey len = 0.074029 GB avg = 0.020 KB total ukey num = 0.003701461 Billion total entry num = 0.003701461 Billion write speed all = 0.020781901 MB/sec (with seq num) write speed all = 0.019686806 MB/sec (without seq num) 2024/11/20-16:18:22.185599 140257248339520 [compaction_job.cc:1948] [default] [JOB 15320] Generated table #44540: 923681 keys, 38788300 bytes, temperature: kWarm 2024/11/20-16:18:22.185659 140257248339520 EVENT_LOG_v1 {"time_micros": 1732090702185629, "cf_name": "default", "job": 15320, "event": "table_file_creation", "file_number": 44540, "file_size": 38788300, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 635932330, "largest_seqno": 763672916, "table_properties": {"data_size": 32825136, "index_size": 1757248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25863068, "raw_average_key_size": 28, "raw_value_size": 108354794, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923681, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732090699, "oldest_key_time": 0, "file_creation_time": 1732090699, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 44540, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-16:18:22.232033 140257248339520 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044541.sst) /dev/shm/ToplingTemp/Topling-561823-746X6A 2024/11/20-16:18:23.048604 140257248339520 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044541.sst): first pass time = 0.82's, 164.377'MB/sec 2024/11/20-16:18:24.163300 140257248339520 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044541.sst): old prealloc_size = 147639500, real_size = 34842672 2024/11/20-16:18:24.163851 140257248339520 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044541.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924334 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0329 GB avg-val = 117.20 avg-zval = 35.54 2024/11/20-16:18:24.164632 140257248339520 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044541.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-16:18:25.001577 140257248339520 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044541.sst): second pass time = 1.11's, 97.268'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.308'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3423.487'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.324'MB/sec write SST data time = 0.12's, 311.069'MB/sec dict compress time = 0.00's, 1081680.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 21716.993'MB/sec zip my value time = 1.11's, unzip length = 0.108'GB zip my value throughput = 97.268'MB/sec zip pipeline throughput = 97.268'MB/sec entries = 924334 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.20 avg-zval = 35.54 usrkeys = 924334 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.20 avg-zval = 35.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924334 bytes = 3119648 } seq expand size = 6470338 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0329 GB dict = 1.08 MB all = 0.0388 GB } UnZip/Zip{ index = 10.5130 value = 3.2976 dict = 1.00 all = 3.4575 } Zip/UnZip{ index = 0.0951 value = 0.3032 dict = 1.00 all = 0.2892 } ---------------------------- total value len = 0.573115 GB avg = 0.124 KB (by entry num) total key len = 0.129522 GB avg = 0.028 KB total ukey len = 0.092516 GB avg = 0.020 KB total ukey num = 0.004625795 Billion total entry num = 0.004625795 Billion write speed all = 0.025982059 MB/sec (with seq num) write speed all = 0.024613640 MB/sec (without seq num) 2024/11/20-16:18:25.047740 140257248339520 [compaction_job.cc:1948] [default] [JOB 15320] Generated table #44541: 924334 keys, 38818784 bytes, temperature: kWarm 2024/11/20-16:18:25.047796 140257248339520 EVENT_LOG_v1 {"time_micros": 1732090705047767, "cf_name": "default", "job": 15320, "event": "table_file_creation", "file_number": 44541, "file_size": 38818784, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 635932356, "largest_seqno": 763672452, "table_properties": {"data_size": 32852960, "index_size": 1758464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25881352, "raw_average_key_size": 28, "raw_value_size": 108336516, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924334, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732090702, "oldest_key_time": 0, "file_creation_time": 1732090702, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 44541, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-16:18:25.083703 140257248339520 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044542.sst) /dev/shm/ToplingTemp/Topling-561823-OkPDtD 2024/11/20-16:18:25.900195 140257248339520 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044542.sst): first pass time = 0.82's, 164.390'MB/sec 2024/11/20-16:18:27.034997 140257248339520 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044542.sst): old prealloc_size = 147639500, real_size = 35535744 2024/11/20-16:18:27.035598 140257248339520 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044542.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927223 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.96 raw-val = 0.1083 GB zip-val = 0.0335 GB avg-val = 116.75 avg-zval = 36.11 2024/11/20-16:18:27.036412 140257248339520 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044542.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-16:18:27.834995 140257248339520 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044542.sst): second pass time = 1.13's, 95.458'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.80's, 49.695'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3105.334'MB/sec (index lex order gen) rebuild zvType time = 0.67's, 0.344'MB/sec write SST data time = 0.13's, 316.946'MB/sec dict compress time = 0.00's, 1080460.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 23538.921'MB/sec zip my value time = 1.13's, unzip length = 0.108'GB zip my value throughput = 95.458'MB/sec zip pipeline throughput = 95.458'MB/sec entries = 927223 avg-key = 28.00 avg-zkey = 1.96 avg-val = 116.75 avg-zval = 36.11 usrkeys = 927223 avg-key = 20.00 avg-zkey = 1.96 avg-val = 123.75 avg-zval = 36.11 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927223 bytes = 3361200 } seq expand size = 6490561 multi value expand size = 0 cnt WriteAppend = 29 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0335 GB dict = 1.08 MB all = 0.0398 GB } UnZip/Zip{ index = 10.1783 value = 3.2333 dict = 1.00 all = 3.3761 } Zip/UnZip{ index = 0.0982 value = 0.3093 dict = 1.00 all = 0.2962 } ---------------------------- total value len = 0.687861 GB avg = 0.124 KB (by entry num) total key len = 0.155485 GB avg = 0.028 KB total ukey len = 0.111060 GB avg = 0.020 KB total ukey num = 0.005553018 Billion total entry num = 0.005553018 Billion write speed all = 0.031181893 MB/sec (with seq num) write speed all = 0.029539353 MB/sec (without seq num) 2024/11/20-16:18:27.882352 140257248339520 [compaction_job.cc:1948] [default] [JOB 15320] Generated table #44542: 927223 keys, 39755100 bytes, temperature: kWarm 2024/11/20-16:18:27.882415 140257248339520 EVENT_LOG_v1 {"time_micros": 1732090707882386, "cf_name": "default", "job": 15320, "event": "table_file_creation", "file_number": 44542, "file_size": 39755100, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 442876808, "largest_seqno": 763673619, "table_properties": {"data_size": 33481776, "index_size": 1821952, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25962244, "raw_average_key_size": 28, "raw_value_size": 108255522, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927223, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732090705, "oldest_key_time": 0, "file_creation_time": 1732090705, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 44542, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-16:18:27.919706 140257248339520 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044543.sst) /dev/shm/ToplingTemp/Topling-561823-iQnrDE 2024/11/20-16:18:28.681073 140257248339520 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044543.sst): first pass time = 0.76's, 176.293'MB/sec 2024/11/20-16:18:29.984920 140257248339520 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044543.sst): old prealloc_size = 147639500, real_size = 36498176 2024/11/20-16:18:29.985550 140257248339520 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044543.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927000 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.79 avg-zval = 37.18 2024/11/20-16:18:29.985797 140257248339520 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044543.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-16:18:30.682369 140257248339520 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044543.sst): second pass time = 1.30's, 83.093'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.70's, 54.475'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3026.051'MB/sec (index lex order gen) rebuild zvType time = 0.57's, 0.406'MB/sec write SST data time = 0.13's, 302.581'MB/sec dict compress time = 0.00's, 1079898.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 19147.807'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.093'MB/sec zip pipeline throughput = 83.093'MB/sec entries = 927000 avg-key = 28.00 avg-zkey = 1.94 avg-val = 116.79 avg-zval = 37.18 usrkeys = 927000 avg-key = 20.00 avg-zkey = 1.94 avg-val = 117.89 avg-zval = 37.18 TagRS{ kinds = RS_Key0_TagN bytes = 144960 } TagArray{ size = 146057 bytes = 492960 } seq expand size = 1022399 multi value expand size = 0 cnt WriteAppend = 47 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.3066 value = 3.1410 dict = 1.00 all = 3.5326 } Zip/UnZip{ index = 0.0970 value = 0.3184 dict = 1.00 all = 0.2831 } ---------------------------- total value len = 0.797145 GB avg = 0.123 KB (by entry num) total key len = 0.181441 GB avg = 0.028 KB total ukey len = 0.129600 GB avg = 0.020 KB total ukey num = 0.006480018 Billion total entry num = 0.006480018 Billion write speed all = 0.036178461 MB/sec (with seq num) write speed all = 0.034261923 MB/sec (without seq num) 2024/11/20-16:18:30.728849 140257248339520 [compaction_job.cc:1948] [default] [JOB 15320] Generated table #44543: 927000 keys, 37993769 bytes, temperature: kWarm 2024/11/20-16:18:30.728905 140257248339520 EVENT_LOG_v1 {"time_micros": 1732090710728876, "cf_name": "default", "job": 15320, "event": "table_file_creation", "file_number": 44543, "file_size": 37993769, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 535975517, "table_properties": {"data_size": 34467376, "index_size": 1798848, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25956000, "raw_average_key_size": 28, "raw_value_size": 108261764, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927000, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732090707, "oldest_key_time": 0, "file_creation_time": 1732090707, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 44543, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-16:18:30.764158 140257248339520 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044544.sst) /dev/shm/ToplingTemp/Topling-561823-ZuQqzE 2024/11/20-16:18:31.520500 140257248339520 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044544.sst): first pass time = 0.76's, 177.463'MB/sec 2024/11/20-16:18:32.804247 140257248339520 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044544.sst): old prealloc_size = 147639500, real_size = 36409152 2024/11/20-16:18:32.804815 140257248339520 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044544.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926892 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1083 GB zip-val = 0.0344 GB avg-val = 116.80 avg-zval = 37.09 2024/11/20-16:18:32.804959 140257248339520 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044544.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-16:18:33.497964 140257248339520 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044544.sst): second pass time = 1.28's, 84.389'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.69's, 53.950'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3307.974'MB/sec (index lex order gen) rebuild zvType time = 0.57's, 0.406'MB/sec write SST data time = 0.12's, 305.001'MB/sec dict compress time = 0.00's, 1080060.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 29445.474'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.389'MB/sec zip pipeline throughput = 84.389'MB/sec entries = 926892 avg-key = 28.00 avg-zkey = 1.94 avg-val = 116.80 avg-zval = 37.09 usrkeys = 926892 avg-key = 20.00 avg-zkey = 1.94 avg-val = 116.83 avg-zval = 37.09 TagRS{ kinds = RS_Key0_TagN bytes = 144920 } TagArray{ size = 3959 bytes = 12880 } seq expand size = 27713 multi value expand size = 0 cnt WriteAppend = 53 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.3069 value = 3.1492 dict = 1.00 all = 3.5863 } Zip/UnZip{ index = 0.0970 value = 0.3175 dict = 1.00 all = 0.2788 } ---------------------------- total value len = 0.905438 GB avg = 0.122 KB (by entry num) total key len = 0.207393 GB avg = 0.028 KB total ukey len = 0.148138 GB avg = 0.020 KB total ukey num = 0.007406910 Billion total entry num = 0.007406910 Billion write speed all = 0.041137259 MB/sec (with seq num) write speed all = 0.038946810 MB/sec (without seq num) 2024/11/20-16:18:33.546653 140257248339520 [compaction_job.cc:1948] [default] [JOB 15320] Generated table #44544: 926892 keys, 37424843 bytes, temperature: kWarm 2024/11/20-16:18:33.546718 140257248339520 EVENT_LOG_v1 {"time_micros": 1732090713546688, "cf_name": "default", "job": 15320, "event": "table_file_creation", "file_number": 44544, "file_size": 37424843, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 538604047, "table_properties": {"data_size": 34378672, "index_size": 1798592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25952976, "raw_average_key_size": 28, "raw_value_size": 108264896, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926892, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732090710, "oldest_key_time": 0, "file_creation_time": 1732090710, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 44544, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-16:18:33.583715 140257248339520 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044545.sst) /dev/shm/ToplingTemp/Topling-561823-RA6V6C 2024/11/20-16:18:34.361772 140257248339520 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044545.sst): first pass time = 0.78's, 172.511'MB/sec 2024/11/20-16:18:35.652829 140257248339520 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044545.sst): old prealloc_size = 147639500, real_size = 36490912 2024/11/20-16:18:35.653454 140257248339520 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044545.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926776 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.82 avg-zval = 37.18 2024/11/20-16:18:35.653806 140257248339520 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044545.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-16:18:36.349671 140257248339520 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044545.sst): second pass time = 1.29's, 83.914'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.70's, 55.339'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2985.927'MB/sec (index lex order gen) rebuild zvType time = 0.57's, 0.406'MB/sec write SST data time = 0.13's, 306.257'MB/sec dict compress time = 0.00's, 1080098.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 20542.775'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.914'MB/sec zip pipeline throughput = 83.914'MB/sec entries = 926776 avg-key = 28.00 avg-zkey = 1.94 avg-val = 116.82 avg-zval = 37.18 usrkeys = 926776 avg-key = 20.00 avg-zkey = 1.94 avg-val = 119.21 avg-zval = 37.18 TagRS{ kinds = RS_Key0_TagN bytes = 144920 } TagArray{ size = 316439 bytes = 1068000 } seq expand size = 2215073 multi value expand size = 0 cnt WriteAppend = 43 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.3067 value = 3.1418 dict = 1.00 all = 3.4806 } Zip/UnZip{ index = 0.0970 value = 0.3183 dict = 1.00 all = 0.2873 } ---------------------------- total value len = 1.015921 GB avg = 0.122 KB (by entry num) total key len = 0.233343 GB avg = 0.028 KB total ukey len = 0.166674 GB avg = 0.020 KB total ukey num = 0.008333686 Billion total entry num = 0.008333686 Billion write speed all = 0.046175812 MB/sec (with seq num) write speed all = 0.043711547 MB/sec (without seq num) 2024/11/20-16:18:36.397013 140257248339520 [compaction_job.cc:1948] [default] [JOB 15320] Generated table #44545: 926776 keys, 38561969 bytes, temperature: kWarm 2024/11/20-16:18:36.397089 140257248339520 EVENT_LOG_v1 {"time_micros": 1732090716397056, "cf_name": "default", "job": 15320, "event": "table_file_creation", "file_number": 44545, "file_size": 38561969, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 541395865, "table_properties": {"data_size": 34460624, "index_size": 1798400, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25949728, "raw_average_key_size": 28, "raw_value_size": 108268084, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926776, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732090713, "oldest_key_time": 0, "file_creation_time": 1732090713, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 44545, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-16:18:36.434416 140257248339520 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044546.sst) /dev/shm/ToplingTemp/Topling-561823-LGM1nB 2024/11/20-16:18:37.232113 140257248339520 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044546.sst): first pass time = 0.80's, 168.266'MB/sec 2024/11/20-16:18:38.411293 140257248339520 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044546.sst): old prealloc_size = 147639500, real_size = 36163872 2024/11/20-16:18:38.411879 140257248339520 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044546.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923724 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.96 raw-val = 0.1084 GB zip-val = 0.0341 GB avg-val = 117.30 avg-zval = 36.94 2024/11/20-16:18:38.412570 140257248339520 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044546.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-16:18:39.168379 140257248339520 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044546.sst): second pass time = 1.18's, 91.959'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.76's, 52.297'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3223.153'MB/sec (index lex order gen) rebuild zvType time = 0.63's, 0.368'MB/sec write SST data time = 0.13's, 308.389'MB/sec dict compress time = 0.00's, 1082522.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32430.258'MB/sec zip my value time = 1.18's, unzip length = 0.108'GB zip my value throughput = 91.959'MB/sec zip pipeline throughput = 91.959'MB/sec entries = 923724 avg-key = 28.00 avg-zkey = 1.96 avg-val = 117.30 avg-zval = 36.94 usrkeys = 923724 avg-key = 20.00 avg-zkey = 1.96 avg-val = 122.55 avg-zval = 36.94 TagRS{ kinds = RS_Key0_TagN bytes = 144440 } TagArray{ size = 692292 bytes = 2423040 } seq expand size = 4846044 multi value expand size = 0 cnt WriteAppend = 39 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0341 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.1796 value = 3.1759 dict = 1.00 all = 3.3900 } Zip/UnZip{ index = 0.0982 value = 0.3149 dict = 1.00 all = 0.2950 } ---------------------------- total value len = 1.129120 GB avg = 0.122 KB (by entry num) total key len = 0.259207 GB avg = 0.028 KB total ukey len = 0.185148 GB avg = 0.020 KB total ukey num = 0.009257410 Billion total entry num = 0.009257410 Billion write speed all = 0.051310600 MB/sec (with seq num) write speed all = 0.048573475 MB/sec (without seq num) 2024/11/20-16:18:39.216067 140257248339520 [compaction_job.cc:1948] [default] [JOB 15320] Generated table #44546: 923724 keys, 39592618 bytes, temperature: kWarm 2024/11/20-16:18:39.216127 140257248339520 EVENT_LOG_v1 {"time_micros": 1732090719216098, "cf_name": "default", "job": 15320, "event": "table_file_creation", "file_number": 44546, "file_size": 39592618, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 742639168, "table_properties": {"data_size": 34117904, "index_size": 1814848, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25864272, "raw_average_key_size": 28, "raw_value_size": 108353496, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923724, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732090716, "oldest_key_time": 0, "file_creation_time": 1732090716, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 44546, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-16:18:39.250633 140257248339520 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044547.sst) /dev/shm/ToplingTemp/Topling-561823-HrafTC 2024/11/20-16:18:40.227644 140257248339520 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044547.sst): first pass time = 0.98's, 137.380'MB/sec 2024/11/20-16:18:41.367152 140257248339520 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044547.sst): old prealloc_size = 147639500, real_size = 36123216 2024/11/20-16:18:41.367681 140257248339520 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044547.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926667 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0341 GB avg-val = 116.84 avg-zval = 36.83 2024/11/20-16:18:41.368518 140257248339520 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044547.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-16:18:42.190134 140257248339520 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044547.sst): second pass time = 1.14's, 95.086'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 48.873'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3487.154'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.333'MB/sec write SST data time = 0.13's, 319.710'MB/sec dict compress time = 0.00's, 1080278.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33044.109'MB/sec zip my value time = 1.14's, unzip length = 0.108'GB zip my value throughput = 95.086'MB/sec zip pipeline throughput = 95.086'MB/sec entries = 926667 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.84 avg-zval = 36.83 usrkeys = 926667 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.84 avg-zval = 36.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926667 bytes = 3243344 } seq expand size = 6486669 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0341 GB dict = 1.08 MB all = 0.0402 GB } UnZip/Zip{ index = 10.5146 value = 3.1724 dict = 1.00 all = 3.3370 } Zip/UnZip{ index = 0.0951 value = 0.3152 dict = 1.00 all = 0.2997 } ---------------------------- total value len = 1.243878 GB avg = 0.122 KB (by entry num) total key len = 0.285154 GB avg = 0.028 KB total ukey len = 0.203682 GB avg = 0.020 KB total ukey num = 0.010184077 Billion total entry num = 0.010184077 Billion write speed all = 0.056504522 MB/sec (with seq num) write speed all = 0.053493747 MB/sec (without seq num) 2024/11/20-16:18:42.238513 140257248339520 [compaction_job.cc:1948] [default] [JOB 15320] Generated table #44547: 926667 keys, 40221030 bytes, temperature: kWarm 2024/11/20-16:18:42.238574 140257248339520 EVENT_LOG_v1 {"time_micros": 1732090722238544, "cf_name": "default", "job": 15320, "event": "table_file_creation", "file_number": 44547, "file_size": 40221030, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 591121923, "largest_seqno": 763674214, "table_properties": {"data_size": 34128768, "index_size": 1762624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25946676, "raw_average_key_size": 28, "raw_value_size": 108271058, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926667, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732090719, "oldest_key_time": 0, "file_creation_time": 1732090719, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 44547, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-16:18:42.282005 140257248339520 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044548.sst) /dev/shm/ToplingTemp/Topling-561823-qPeI7A 2024/11/20-16:18:43.090700 140257248339520 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044548.sst): first pass time = 0.81's, 165.975'MB/sec 2024/11/20-16:18:44.215650 140257248339520 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044548.sst): old prealloc_size = 147639500, real_size = 36294592 2024/11/20-16:18:44.216252 140257248339520 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044548.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926191 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0343 GB avg-val = 116.91 avg-zval = 37.01 2024/11/20-16:18:44.217020 140257248339520 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044548.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-16:18:45.031726 140257248339520 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044548.sst): second pass time = 1.12's, 96.333'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 49.353'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3148.718'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 320.208'MB/sec dict compress time = 0.00's, 1080458.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 17227.789'MB/sec zip my value time = 1.12's, unzip length = 0.108'GB zip my value throughput = 96.333'MB/sec zip pipeline throughput = 96.333'MB/sec entries = 926191 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.91 avg-zval = 37.01 usrkeys = 926191 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.91 avg-zval = 37.01 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926191 bytes = 3125904 } seq expand size = 6483337 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0343 GB dict = 1.08 MB all = 0.0403 GB } UnZip/Zip{ index = 10.3762 value = 3.1590 dict = 1.00 all = 3.3325 } Zip/UnZip{ index = 0.0964 value = 0.3166 dict = 1.00 all = 0.3001 } ---------------------------- total value len = 1.358646 GB avg = 0.122 KB (by entry num) total key len = 0.311088 GB avg = 0.028 KB total ukey len = 0.222205 GB avg = 0.020 KB total ukey num = 0.011110268 Billion total entry num = 0.011110268 Billion write speed all = 0.061697574 MB/sec (with seq num) write speed all = 0.058413330 MB/sec (without seq num) 2024/11/20-16:18:45.079350 140257248339520 [compaction_job.cc:1948] [default] [JOB 15320] Generated table #44548: 926191 keys, 40275162 bytes, temperature: kWarm 2024/11/20-16:18:45.079412 140257248339520 EVENT_LOG_v1 {"time_micros": 1732090725079382, "cf_name": "default", "job": 15320, "event": "table_file_creation", "file_number": 44548, "file_size": 40275162, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 632970998, "largest_seqno": 763674270, "table_properties": {"data_size": 34277616, "index_size": 1785216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25933348, "raw_average_key_size": 28, "raw_value_size": 108284466, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926191, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732090722, "oldest_key_time": 0, "file_creation_time": 1732090722, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 44548, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-16:18:45.113553 140257248339520 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044549.sst) /dev/shm/ToplingTemp/Topling-561823-Cuk5mE 2024/11/20-16:18:45.888451 140257248339520 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044549.sst): first pass time = 0.77's, 173.214'MB/sec 2024/11/20-16:18:47.142079 140257248339520 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044549.sst): old prealloc_size = 147639500, real_size = 36682336 2024/11/20-16:18:47.142627 140257248339520 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044549.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927287 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1083 GB zip-val = 0.0347 GB avg-val = 116.74 avg-zval = 37.37 2024/11/20-16:18:47.142935 140257248339520 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044549.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-16:18:47.858528 140257248339520 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044549.sst): second pass time = 1.25's, 86.413'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.72's, 53.778'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3442.247'MB/sec (index lex order gen) rebuild zvType time = 0.59's, 0.393'MB/sec write SST data time = 0.13's, 306.814'MB/sec dict compress time = 0.00's, 1080310.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 18701.485'MB/sec zip my value time = 1.25's, unzip length = 0.108'GB zip my value throughput = 86.413'MB/sec zip pipeline throughput = 86.413'MB/sec entries = 927287 avg-key = 28.00 avg-zkey = 1.94 avg-val = 116.74 avg-zval = 37.37 usrkeys = 927287 avg-key = 20.00 avg-zkey = 1.94 avg-val = 118.50 avg-zval = 37.37 TagRS{ kinds = RS_Key0_TagN bytes = 145000 } TagArray{ size = 233004 bytes = 844656 } seq expand size = 1631028 multi value expand size = 0 cnt WriteAppend = 22 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0347 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.3083 value = 3.1241 dict = 1.00 all = 3.4835 } Zip/UnZip{ index = 0.0970 value = 0.3201 dict = 1.00 all = 0.2871 } ---------------------------- total value len = 1.468531 GB avg = 0.122 KB (by entry num) total key len = 0.337052 GB avg = 0.028 KB total ukey len = 0.240751 GB avg = 0.020 KB total ukey num = 0.012037555 Billion total entry num = 0.012037555 Billion write speed all = 0.066710298 MB/sec (with seq num) write speed all = 0.063152315 MB/sec (without seq num) 2024/11/20-16:18:47.904948 140257248339520 [compaction_job.cc:1948] [default] [JOB 15320] Generated table #44549: 927287 keys, 38530053 bytes, temperature: kWarm 2024/11/20-16:18:47.905009 140257248339520 EVENT_LOG_v1 {"time_micros": 1732090727904980, "cf_name": "default", "job": 15320, "event": "table_file_creation", "file_number": 44549, "file_size": 38530053, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 763674282, "table_properties": {"data_size": 34651216, "index_size": 1799104, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25964036, "raw_average_key_size": 28, "raw_value_size": 108253822, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927287, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732090725, "oldest_key_time": 0, "file_creation_time": 1732090725, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 44549, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-16:18:47.942670 140257248339520 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044550.sst) /dev/shm/ToplingTemp/Topling-561823-UykPLE 2024/11/20-16:18:48.752341 140257248339520 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044550.sst): first pass time = 0.81's, 165.775'MB/sec 2024/11/20-16:18:49.892703 140257248339520 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044550.sst): old prealloc_size = 147639500, real_size = 33107248 2024/11/20-16:18:49.893258 140257248339520 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044550.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922952 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0311 GB avg-val = 117.42 avg-zval = 33.72 2024/11/20-16:18:49.894090 140257248339520 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044550.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-16:18:50.703901 140257248339520 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044550.sst): second pass time = 1.14's, 95.100'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 46.000'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3301.177'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.337'MB/sec write SST data time = 0.13's, 298.036'MB/sec dict compress time = 0.00's, 1082736.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 27725.494'MB/sec zip my value time = 1.14's, unzip length = 0.108'GB zip my value throughput = 95.100'MB/sec zip pipeline throughput = 95.100'MB/sec entries = 922952 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.42 avg-zval = 33.72 usrkeys = 922952 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.42 avg-zval = 33.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922952 bytes = 3345712 } seq expand size = 6460664 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5129 value = 3.4824 dict = 1.00 all = 3.5969 } Zip/UnZip{ index = 0.0951 value = 0.2872 dict = 1.00 all = 0.2780 } ---------------------------- total value len = 1.583367 GB avg = 0.122 KB (by entry num) total key len = 0.362894 GB avg = 0.028 KB total ukey len = 0.259210 GB avg = 0.020 KB total ukey num = 0.012960507 Billion total entry num = 0.012960507 Billion write speed all = 0.071900343 MB/sec (with seq num) write speed all = 0.068069963 MB/sec (without seq num) 2024/11/20-16:18:50.748204 140257248339520 [compaction_job.cc:1948] [default] [JOB 15320] Generated table #44550: 922952 keys, 37314432 bytes, temperature: kWarm 2024/11/20-16:18:50.748265 140257248339520 EVENT_LOG_v1 {"time_micros": 1732090730748234, "cf_name": "default", "job": 15320, "event": "table_file_creation", "file_number": 44550, "file_size": 37314432, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 491642472, "largest_seqno": 763673191, "table_properties": {"data_size": 31120480, "index_size": 1755840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25842656, "raw_average_key_size": 28, "raw_value_size": 108375124, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922952, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732090727, "oldest_key_time": 0, "file_creation_time": 1732090727, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 44550, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-16:18:50.788641 140257248339520 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044551.sst) /dev/shm/ToplingTemp/Topling-561823-BtzigB 2024/11/20-16:18:51.595702 140257248339520 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044551.sst): first pass time = 0.81's, 166.310'MB/sec 2024/11/20-16:18:52.737273 140257248339520 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044551.sst): old prealloc_size = 147639500, real_size = 35147728 2024/11/20-16:18:52.737799 140257248339520 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044551.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925026 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0332 GB avg-val = 117.10 avg-zval = 35.84 2024/11/20-16:18:52.738593 140257248339520 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044551.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-16:18:53.570807 140257248339520 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044551.sst): second pass time = 1.14's, 94.962'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 46.939'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3597.700'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 310.286'MB/sec dict compress time = 0.00's, 1080834.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 17854.991'MB/sec zip my value time = 1.14's, unzip length = 0.108'GB zip my value throughput = 94.962'MB/sec zip pipeline throughput = 94.962'MB/sec entries = 925026 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.10 avg-zval = 35.84 usrkeys = 925026 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.10 avg-zval = 35.84 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925026 bytes = 3121984 } seq expand size = 6475182 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0391 GB } UnZip/Zip{ index = 10.5128 value = 3.2668 dict = 1.00 all = 3.4305 } Zip/UnZip{ index = 0.0951 value = 0.3061 dict = 1.00 all = 0.2915 } ---------------------------- total value len = 1.698159 GB avg = 0.122 KB (by entry num) total key len = 0.388795 GB avg = 0.028 KB total ukey len = 0.277711 GB avg = 0.020 KB total ukey num = 0.013885533 Billion total entry num = 0.013885533 Billion write speed all = 0.077089773 MB/sec (with seq num) write speed all = 0.072986443 MB/sec (without seq num) 2024/11/20-16:18:53.618154 140257248339520 [compaction_job.cc:1948] [default] [JOB 15320] Generated table #44551: 925026 keys, 39125138 bytes, temperature: kWarm 2024/11/20-16:18:53.618216 140257248339520 EVENT_LOG_v1 {"time_micros": 1732090733618186, "cf_name": "default", "job": 15320, "event": "table_file_creation", "file_number": 44551, "file_size": 39125138, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 664578877, "largest_seqno": 763674076, "table_properties": {"data_size": 33156480, "index_size": 1759808, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25900728, "raw_average_key_size": 28, "raw_value_size": 108317088, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925026, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732090730, "oldest_key_time": 0, "file_creation_time": 1732090730, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 44551, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-16:18:53.655258 140257248339520 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044552.sst) /dev/shm/ToplingTemp/Topling-561823-1FtKEC 2024/11/20-16:18:54.106254 140257248339520 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044552.sst): first pass time = 0.45's, 264.721'MB/sec 2024/11/20-16:18:55.136653 140257248339520 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044552.sst): old prealloc_size = 147639500, real_size = 31910864 2024/11/20-16:18:55.137156 140257248339520 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044552.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 825719 prefix = 4 raw-key = 0.0132 GB zip-key = 0.0016 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0963 GB zip-val = 0.0301 GB avg-val = 116.58 avg-zval = 36.49 2024/11/20-16:18:55.137854 140257248339520 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044552.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0021 GB, waited 0.000 sec, Key+Value = 0.119 GB 2024/11/20-16:18:55.875257 140257248339520 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00/044552.sst): second pass time = 1.03's, 93.472'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.74's, 48.007'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3240.812'MB/sec (index lex order gen) rebuild zvType time = 0.62's, 0.330'MB/sec write SST data time = 0.11's, 313.173'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.961'MB, throughput = 18358.182'MB/sec zip my value time = 1.03's, unzip length = 0.096'GB zip my value throughput = 93.472'MB/sec zip pipeline throughput = 93.472'MB/sec entries = 825719 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.58 avg-zval = 36.49 usrkeys = 825719 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.58 avg-zval = 36.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 825719 bytes = 2786816 } seq expand size = 5780033 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0165 GB value = 0.0963 GB dict = 0.96 MB all = 0.1194 GB } __ZipSize{ index = 0.0016 GB value = 0.0301 GB dict = 0.96 MB all = 0.0355 GB } UnZip/Zip{ index = 10.5111 value = 3.1945 dict = 1.00 all = 3.3668 } Zip/UnZip{ index = 0.0951 value = 0.3130 dict = 1.00 all = 0.2970 } ---------------------------- total value len = 1.800198 GB avg = 0.122 KB (by entry num) total key len = 0.411915 GB avg = 0.028 KB total ukey len = 0.294225 GB avg = 0.020 KB total ukey num = 0.014711252 Billion total entry num = 0.014711252 Billion write speed all = 0.081706078 MB/sec (with seq num) write speed all = 0.077359108 MB/sec (without seq num) 2024/11/20-16:18:55.918980 140257248339520 [compaction_job.cc:1948] [default] [JOB 15320] Generated table #44552: 825719 keys, 35458010 bytes, temperature: kWarm 2024/11/20-16:18:55.919028 140257248339520 EVENT_LOG_v1 {"time_micros": 1732090735918999, "cf_name": "default", "job": 15320, "event": "table_file_creation", "file_number": 44552, "file_size": 35458010, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 664578947, "largest_seqno": 763673425, "table_properties": {"data_size": 30133120, "index_size": 1571136, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 23120132, "raw_average_key_size": 28, "raw_value_size": 96259482, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 825719, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732090733, "oldest_key_time": 0, "file_creation_time": 1732090733, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 44552, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-16:18:55.945030 140257248339520 [compaction_job.cc:703] job-15320: subcompact[0], size: 0.619122 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15320/att-00 [044537,044538,044539,044540,044541,044542,044543,044544,044545,044546,044547,044548,044549,044550,044551,044552] 2024/11/20-16:18:56.041936 140257248339520 (Original Log Time 2024/11/20-16:18:10.110483) EVENT_LOG_v1 {"time_micros": 1732090690110447, "job": 15320, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [44496], "files_L3": [44488, 44489, 44422, 44423, 44424, 44425, 44426, 44427, 44428, 44429, 44430, 44431, 44316, 44317, 44318, 44319, 44526], "score": 0.947335, "input_data_size": 667162412, "oldest_snapshot_seqno": 764392848} 2024/11/20-16:18:56.041939 140257248339520 (Original Log Time 2024/11/20-16:18:56.040607) [compaction_job.cc:2022] [default] [JOB 15320] Compacted 1@2 + 17@3 files to L3 => 619121576 bytes 2024/11/20-16:18:56.041940 140257248339520 (Original Log Time 2024/11/20-16:18:56.041873) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 16 0 0 0] max score 0.02, MB/sec: 14.6 rd, 13.5 wr, level 3, files in(1, 17) out(16 +0 blob) MB in(54.3, 581.9 +0.0 blob) out(590.4 +0.0 blob), read-write-amplify(22.6) write-amplify(10.9) OK, records in: 14901280, records dropped: 190028 output_compression: Snappy 2024/11/20-16:18:56.041944 140257248339520 (Original Log Time 2024/11/20-16:18:56.041924) EVENT_LOG_v1 {"time_micros": 1732090736041891, "cf": "default", "job": 15320, "event": "compaction_finished", "compaction_time_micros": 45834552, "compaction_time_cpu_micros": 20535868, "output_level": 3, "num_output_files": 16, "total_output_size": 619121576, "num_input_records": 14901280, "num_output_records": 14711252, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 408122890, "file_range_sync_nanos": 0, "file_fsync_nanos": 14199, "file_prepare_write_nanos": 40951, "lsm_state": [0, 0, 0, 16, 0, 0, 0]} 2024/11/20-16:18:56.182896 140257248339520 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732076400,"starts":1732090689,"executesMs":46092,"compactionJobId":15320,"attempt":0,"compactionInputRawBytes":2159630792,"compactionInputZipBytes":667162412,"compactionOutputRawBytes":0,"compactionOutputZipBytes":619121576,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-234e9cc1f29316d328a8932fe9376e27-a3759bcefd03fed1-00"}}, response = 2024/11/20-16:18:56.182970 140257248339520 INFO dcompact_worker.cpp:1464: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T12.20.00/.rocksdb/job-15320/att-00: olev 3, work 46.093 s, result 6.007 ms, install 1.350 ms, input{raw 2.011 GiB zip 636.256 MiB}