2024/11/23-03:09:44.015636 140025171461696 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00: fork to child time = 0.022629 sec 2024/11/23-03:09:44.016715 140025171461696 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00 2024/11/23-03:09:44.025483 140025171461696 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00 2024/11/23-03:09:44.026936 140025171461696 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-03:09:44.027309 140025171461696 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-03:09:44.084878 140025171461696 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-18051/att-00/MANIFEST-206347 2024/11/23-03:09:44.116594 140025171461696 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-18051/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 242103, last_sequence is 3751878826, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 241950 2024/11/23-03:09:44.116607 140025171461696 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-03:09:44.116845 140025171461696 [version_set.cc:5721] Creating manifest 242103 2024/11/23-03:09:44.157026 140025171461696 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-03:09:44.183295 140025171461696 [compaction_job.cc:2437] [default] [JOB 18051] Compacting 1@2 + 11@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-03:09:44.183320 140025171461696 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 13484 Base level 2, inputs: [242098(25MB)], [242019(35MB) 242022(36MB) 242023(37MB) 242024(35MB) 242025(35MB) 242039(35MB) 242040(35MB) 242041(35MB) 242042(34MB) 242043(35MB) 242044(10MB)] 2024/11/23-03:09:44.190886 140025171461696 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-03:09:44.190947 140025171461696 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-03:09:44.190976 140025171461696 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-03:09:44.190988 140025171461696 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-03:09:44.191002 140025171461696 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-03:09:44.747712 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242104.sst) /dev/shm/ToplingTemp/Topling-390667-Nv0gpB 2024/11/23-03:09:45.819042 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242104.sst): first pass time = 1.07's, 125.288'MB/sec 2024/11/23-03:09:47.763929 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242104.sst): old prealloc_size = 147639500, real_size = 33132848 2024/11/23-03:09:47.764493 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242104.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933110 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0311 GB avg-val = 115.84 avg-zval = 33.33 2024/11/23-03:09:47.765285 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242104.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:09:48.885946 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242104.sst): second pass time = 1.94's, 55.615'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.12's, 33.303'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3306.477'MB/sec (index lex order gen) rebuild zvType time = 0.96's, 0.242'MB/sec write SST data time = 0.16's, 238.214'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34589.213'MB/sec zip my value time = 1.94's, unzip length = 0.108'GB zip my value throughput = 55.615'MB/sec zip pipeline throughput = 112.583'MB/sec entries = 933110 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.84 avg-zval = 33.33 usrkeys = 933110 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.84 avg-zval = 33.33 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933110 bytes = 3382544 } seq expand size = 6531770 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.3742 value = 3.4755 dict = 1.00 all = 3.5920 } Zip/UnZip{ index = 0.0964 value = 0.2877 dict = 1.00 all = 0.2784 } ---------------------------- total value len = 0.114623 GB avg = 0.123 KB (by entry num) total key len = 0.026127 GB avg = 0.028 KB total ukey len = 0.018662 GB avg = 0.020 KB total ukey num = 0.000933110 Billion total entry num = 0.000933110 Billion write speed all = 0.002137564 MB/sec (with seq num) write speed all = 0.002024195 MB/sec (without seq num) 2024/11/23-03:09:48.923696 140025171461696 [compaction_job.cc:1948] [default] [JOB 18051] Generated table #242104: 933110 keys, 37366058 bytes, temperature: kWarm 2024/11/23-03:09:48.923818 140025171461696 EVENT_LOG_v1 {"time_micros": 1732302588923737, "cf_name": "default", "job": 18051, "event": "table_file_creation", "file_number": 242104, "file_size": 37366058, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3339428666, "largest_seqno": 3751819416, "table_properties": {"data_size": 31100448, "index_size": 1798912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26127080, "raw_average_key_size": 28, "raw_value_size": 108090772, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933110, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732302584, "oldest_key_time": 0, "file_creation_time": 1732302584, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 242104, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:09:48.999097 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242105.sst) /dev/shm/ToplingTemp/Topling-390667-BkITHD 2024/11/23-03:09:50.064621 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242105.sst): first pass time = 1.07's, 125.970'MB/sec 2024/11/23-03:09:52.195059 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242105.sst): old prealloc_size = 147639500, real_size = 34338240 2024/11/23-03:09:52.195731 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242105.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927649 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0323 GB avg-val = 116.69 avg-zval = 34.85 2024/11/23-03:09:52.196556 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242105.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:09:53.436916 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242105.sst): second pass time = 2.13's, 50.827'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.24's, 31.047'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2742.516'MB/sec (index lex order gen) rebuild zvType time = 1.08's, 0.214'MB/sec write SST data time = 0.16's, 246.459'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34315.278'MB/sec zip my value time = 2.13's, unzip length = 0.108'GB zip my value throughput = 50.827'MB/sec zip pipeline throughput = 85.225'MB/sec entries = 927649 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.69 avg-zval = 34.85 usrkeys = 927649 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.69 avg-zval = 34.85 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927649 bytes = 3362736 } seq expand size = 6493543 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0323 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.4416 value = 3.3482 dict = 1.00 all = 3.4812 } Zip/UnZip{ index = 0.0958 value = 0.2987 dict = 1.00 all = 0.2873 } ---------------------------- total value len = 0.229360 GB avg = 0.123 KB (by entry num) total key len = 0.052101 GB avg = 0.028 KB total ukey len = 0.037215 GB avg = 0.020 KB total ukey num = 0.001860759 Billion total entry num = 0.001860759 Billion write speed all = 0.004274252 MB/sec (with seq num) write speed all = 0.004048192 MB/sec (without seq num) 2024/11/23-03:09:53.471533 140025171461696 [compaction_job.cc:1948] [default] [JOB 18051] Generated table #242105: 927649 keys, 38555102 bytes, temperature: kWarm 2024/11/23-03:09:53.471589 140025171461696 EVENT_LOG_v1 {"time_micros": 1732302593471559, "cf_name": "default", "job": 18051, "event": "table_file_creation", "file_number": 242105, "file_size": 38555102, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 3751819219, "table_properties": {"data_size": 32329328, "index_size": 1776832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25974172, "raw_average_key_size": 28, "raw_value_size": 108243586, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927649, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732302588, "oldest_key_time": 0, "file_creation_time": 1732302588, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 242105, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:09:53.520095 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242106.sst) /dev/shm/ToplingTemp/Topling-390667-pm7f8C 2024/11/23-03:09:54.680918 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242106.sst): first pass time = 1.16's, 115.628'MB/sec 2024/11/23-03:09:56.960131 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242106.sst): old prealloc_size = 147639500, real_size = 34916544 2024/11/23-03:09:56.960692 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242106.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931145 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0329 GB avg-val = 116.14 avg-zval = 35.33 2024/11/23-03:09:56.961401 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242106.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:09:57.949621 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242106.sst): second pass time = 2.28's, 47.463'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.99's, 39.041'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3298.677'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.306'MB/sec write SST data time = 0.23's, 169.412'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31555.107'MB/sec zip my value time = 2.28's, unzip length = 0.108'GB zip my value throughput = 47.463'MB/sec zip pipeline throughput = 89.999'MB/sec entries = 931145 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.14 avg-zval = 35.33 usrkeys = 931145 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.77 avg-zval = 35.33 TagRS{ kinds = RS_Key0_TagN bytes = 145600 } TagArray{ size = 748148 bytes = 2712048 } seq expand size = 5237036 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0329 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.4452 value = 3.2870 dict = 1.00 all = 3.4745 } Zip/UnZip{ index = 0.0957 value = 0.3042 dict = 1.00 all = 0.2878 } ---------------------------- total value len = 0.342742 GB avg = 0.123 KB (by entry num) total key len = 0.078173 GB avg = 0.028 KB total ukey len = 0.055838 GB avg = 0.020 KB total ukey num = 0.002791904 Billion total entry num = 0.002791904 Billion write speed all = 0.006391567 MB/sec (with seq num) write speed all = 0.006052409 MB/sec (without seq num) 2024/11/23-03:09:57.989562 140025171461696 [compaction_job.cc:1948] [default] [JOB 18051] Generated table #242106: 931145 keys, 38629166 bytes, temperature: kWarm 2024/11/23-03:09:57.989788 140025171461696 EVENT_LOG_v1 {"time_micros": 1732302597989599, "cf_name": "default", "job": 18051, "event": "table_file_creation", "file_number": 242106, "file_size": 38629166, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3751820199, "table_properties": {"data_size": 32900656, "index_size": 1782912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26072060, "raw_average_key_size": 28, "raw_value_size": 108145706, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931145, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732302593, "oldest_key_time": 0, "file_creation_time": 1732302593, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 242106, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:09:58.040028 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242107.sst) /dev/shm/ToplingTemp/Topling-390667-PTfQpD 2024/11/23-03:09:58.951419 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242107.sst): first pass time = 0.91's, 147.275'MB/sec 2024/11/23-03:10:00.467381 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242107.sst): old prealloc_size = 147639500, real_size = 32956688 2024/11/23-03:10:00.467976 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242107.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927223 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 116.75 avg-zval = 33.39 2024/11/23-03:10:00.468705 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242107.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:10:01.388640 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242107.sst): second pass time = 1.51's, 71.476'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.224'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3087.713'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.293'MB/sec write SST data time = 0.13's, 287.873'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33368.746'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.476'MB/sec zip pipeline throughput = 91.178'MB/sec entries = 927223 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.75 avg-zval = 33.39 usrkeys = 927223 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.75 avg-zval = 33.39 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927223 bytes = 3245296 } seq expand size = 6490561 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5141 value = 3.4965 dict = 1.00 all = 3.6220 } Zip/UnZip{ index = 0.0951 value = 0.2860 dict = 1.00 all = 0.2761 } ---------------------------- total value len = 0.457489 GB avg = 0.123 KB (by entry num) total key len = 0.104136 GB avg = 0.028 KB total ukey len = 0.074383 GB avg = 0.020 KB total ukey num = 0.003719127 Billion total entry num = 0.003719127 Billion write speed all = 0.008527766 MB/sec (with seq num) write speed all = 0.008075993 MB/sec (without seq num) 2024/11/23-03:10:01.425203 140025171461696 [compaction_job.cc:1948] [default] [JOB 18051] Generated table #242107: 927223 keys, 37056432 bytes, temperature: kWarm 2024/11/23-03:10:01.425267 140025171461696 EVENT_LOG_v1 {"time_micros": 1732302601425234, "cf_name": "default", "job": 18051, "event": "table_file_creation", "file_number": 242107, "file_size": 37056432, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 3751819499, "table_properties": {"data_size": 30960896, "index_size": 1763776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25962244, "raw_average_key_size": 28, "raw_value_size": 108255574, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927223, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732302598, "oldest_key_time": 0, "file_creation_time": 1732302598, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 242107, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:10:01.466210 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242108.sst) /dev/shm/ToplingTemp/Topling-390667-IQBmwz 2024/11/23-03:10:02.301032 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242108.sst): first pass time = 0.83's, 160.781'MB/sec 2024/11/23-03:10:03.794462 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242108.sst): old prealloc_size = 147639500, real_size = 33159648 2024/11/23-03:10:03.795034 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242108.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926702 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 116.83 avg-zval = 33.63 2024/11/23-03:10:03.795802 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242108.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:10:04.749748 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242108.sst): second pass time = 1.49's, 72.531'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 39.002'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3221.500'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.281'MB/sec write SST data time = 0.13's, 286.630'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34160.974'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.531'MB/sec zip pipeline throughput = 99.255'MB/sec entries = 926702 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.83 avg-zval = 33.63 usrkeys = 926702 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.83 avg-zval = 33.63 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926702 bytes = 3243472 } seq expand size = 6486914 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5127 value = 3.4741 dict = 1.00 all = 3.6024 } Zip/UnZip{ index = 0.0951 value = 0.2878 dict = 1.00 all = 0.2776 } ---------------------------- total value len = 0.572246 GB avg = 0.123 KB (by entry num) total key len = 0.130083 GB avg = 0.028 KB total ukey len = 0.092917 GB avg = 0.020 KB total ukey num = 0.004645829 Billion total entry num = 0.004645829 Billion write speed all = 0.010663699 MB/sec (with seq num) write speed all = 0.010099385 MB/sec (without seq num) 2024/11/23-03:10:04.796302 140025171461696 [compaction_job.cc:1948] [default] [JOB 18051] Generated table #242108: 926702 keys, 37257402 bytes, temperature: kWarm 2024/11/23-03:10:04.796365 140025171461696 EVENT_LOG_v1 {"time_micros": 1732302604796333, "cf_name": "default", "job": 18051, "event": "table_file_creation", "file_number": 242108, "file_size": 37257402, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3604582272, "largest_seqno": 3751819661, "table_properties": {"data_size": 31164752, "index_size": 1763008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25947656, "raw_average_key_size": 28, "raw_value_size": 108270168, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926702, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732302601, "oldest_key_time": 0, "file_creation_time": 1732302601, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 242108, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:10:04.837635 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242109.sst) /dev/shm/ToplingTemp/Topling-390667-t8EpXD 2024/11/23-03:10:05.640019 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242109.sst): first pass time = 0.80's, 167.284'MB/sec 2024/11/23-03:10:07.115873 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242109.sst): old prealloc_size = 147639500, real_size = 33353840 2024/11/23-03:10:07.116396 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242109.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927200 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.76 avg-zval = 33.81 2024/11/23-03:10:07.117166 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242109.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:10:08.074499 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242109.sst): second pass time = 1.48's, 73.383'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 39.071'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3520.706'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.279'MB/sec write SST data time = 0.13's, 291.796'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33918.689'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.383'MB/sec zip pipeline throughput = 77.582'MB/sec entries = 927200 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.76 avg-zval = 33.81 usrkeys = 927200 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.76 avg-zval = 33.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927200 bytes = 3245216 } seq expand size = 6490400 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4441 value = 3.4536 dict = 1.00 all = 3.5836 } Zip/UnZip{ index = 0.0957 value = 0.2896 dict = 1.00 all = 0.2791 } ---------------------------- total value len = 0.686992 GB avg = 0.123 KB (by entry num) total key len = 0.156045 GB avg = 0.028 KB total ukey len = 0.111461 GB avg = 0.020 KB total ukey num = 0.005573029 Billion total entry num = 0.005573029 Billion write speed all = 0.012799474 MB/sec (with seq num) write speed all = 0.012122571 MB/sec (without seq num) 2024/11/23-03:10:08.105398 140025171461696 [compaction_job.cc:1948] [default] [JOB 18051] Generated table #242109: 927200 keys, 37453548 bytes, temperature: kWarm 2024/11/23-03:10:08.105458 140025171461696 EVENT_LOG_v1 {"time_micros": 1732302608105427, "cf_name": "default", "job": 18051, "event": "table_file_creation", "file_number": 242109, "file_size": 37453548, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3485447251, "largest_seqno": 3751820156, "table_properties": {"data_size": 31346336, "index_size": 1775552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25961600, "raw_average_key_size": 28, "raw_value_size": 108256240, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927200, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732302604, "oldest_key_time": 0, "file_creation_time": 1732302604, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 242109, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:10:08.143120 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242110.sst) /dev/shm/ToplingTemp/Topling-390667-od7ksA 2024/11/23-03:10:08.990261 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242110.sst): first pass time = 0.85's, 158.443'MB/sec 2024/11/23-03:10:10.633167 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242110.sst): old prealloc_size = 147639500, real_size = 33402208 2024/11/23-03:10:10.633714 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242110.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927151 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.76 avg-zval = 33.86 2024/11/23-03:10:10.634492 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242110.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:10:11.591488 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242110.sst): second pass time = 1.64's, 65.921'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 39.254'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3373.035'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.279'MB/sec write SST data time = 0.13's, 295.312'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 27275.876'MB/sec zip my value time = 1.64's, unzip length = 0.108'GB zip my value throughput = 65.921'MB/sec zip pipeline throughput = 129.125'MB/sec entries = 927151 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.76 avg-zval = 33.86 usrkeys = 927151 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.76 avg-zval = 33.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927151 bytes = 3360944 } seq expand size = 6490057 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.4435 value = 3.4483 dict = 1.00 all = 3.5680 } Zip/UnZip{ index = 0.0958 value = 0.2900 dict = 1.00 all = 0.2803 } ---------------------------- total value len = 0.801740 GB avg = 0.123 KB (by entry num) total key len = 0.182005 GB avg = 0.028 KB total ukey len = 0.130004 GB avg = 0.020 KB total ukey num = 0.006500180 Billion total entry num = 0.006500180 Billion write speed all = 0.014934985 MB/sec (with seq num) write speed all = 0.014145512 MB/sec (without seq num) 2024/11/23-03:10:11.623360 140025171461696 [compaction_job.cc:1948] [default] [JOB 18051] Generated table #242110: 927151 keys, 37617556 bytes, temperature: kWarm 2024/11/23-03:10:11.623422 140025171461696 EVENT_LOG_v1 {"time_micros": 1732302611623390, "cf_name": "default", "job": 18051, "event": "table_file_creation", "file_number": 242110, "file_size": 37617556, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268533, "largest_seqno": 3751819999, "table_properties": {"data_size": 31394704, "index_size": 1775552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25960228, "raw_average_key_size": 28, "raw_value_size": 108257642, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927151, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732302608, "oldest_key_time": 0, "file_creation_time": 1732302608, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 242110, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:10:11.661808 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242111.sst) /dev/shm/ToplingTemp/Topling-390667-PIihBC 2024/11/23-03:10:12.473756 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242111.sst): first pass time = 0.81's, 165.311'MB/sec 2024/11/23-03:10:13.927891 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242111.sst): old prealloc_size = 147639500, real_size = 33372496 2024/11/23-03:10:13.928402 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242111.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933125 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0314 GB avg-val = 115.84 avg-zval = 33.60 2024/11/23-03:10:13.929161 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242111.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:10:14.860726 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242111.sst): second pass time = 1.45's, 74.395'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.189'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3647.926'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.289'MB/sec write SST data time = 0.13's, 297.667'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36341.482'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.395'MB/sec zip pipeline throughput = 74.395'MB/sec entries = 933125 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.84 avg-zval = 33.60 usrkeys = 933125 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.84 avg-zval = 33.60 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933125 bytes = 3265952 } seq expand size = 6531875 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4438 value = 3.4476 dict = 1.00 all = 3.5802 } Zip/UnZip{ index = 0.0958 value = 0.2901 dict = 1.00 all = 0.2793 } ---------------------------- total value len = 0.916362 GB avg = 0.123 KB (by entry num) total key len = 0.208133 GB avg = 0.028 KB total ukey len = 0.148666 GB avg = 0.020 KB total ukey num = 0.007433305 Billion total entry num = 0.007433305 Billion write speed all = 0.017070967 MB/sec (with seq num) write speed all = 0.016168206 MB/sec (without seq num) 2024/11/23-03:10:14.893624 140025171461696 [compaction_job.cc:1948] [default] [JOB 18051] Generated table #242111: 933125 keys, 37488876 bytes, temperature: kWarm 2024/11/23-03:10:14.893685 140025171461696 EVENT_LOG_v1 {"time_micros": 1732302614893654, "cf_name": "default", "job": 18051, "event": "table_file_creation", "file_number": 242111, "file_size": 37488876, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 3751819834, "table_properties": {"data_size": 31352064, "index_size": 1786944, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26127500, "raw_average_key_size": 28, "raw_value_size": 108090346, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933125, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732302611, "oldest_key_time": 0, "file_creation_time": 1732302611, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 242111, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:10:14.937510 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242112.sst) /dev/shm/ToplingTemp/Topling-390667-UZVWBD 2024/11/23-03:10:15.701595 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242112.sst): first pass time = 0.76's, 175.669'MB/sec 2024/11/23-03:10:17.153350 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242112.sst): old prealloc_size = 147639500, real_size = 32818288 2024/11/23-03:10:17.153943 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242112.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923080 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0308 GB avg-val = 117.40 avg-zval = 33.40 2024/11/23-03:10:17.154728 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242112.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:10:18.087452 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242112.sst): second pass time = 1.45's, 74.681'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 39.634'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3070.957'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.293'MB/sec write SST data time = 0.15's, 254.594'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 33539.086'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.681'MB/sec zip pipeline throughput = 85.959'MB/sec entries = 923080 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.40 avg-zval = 33.40 usrkeys = 923080 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.40 avg-zval = 33.40 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923080 bytes = 3346176 } seq expand size = 6461560 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5129 value = 3.5150 dict = 1.00 all = 3.6253 } Zip/UnZip{ index = 0.0951 value = 0.2845 dict = 1.00 all = 0.2758 } ---------------------------- total value len = 1.031195 GB avg = 0.123 KB (by entry num) total key len = 0.233979 GB avg = 0.028 KB total ukey len = 0.167128 GB avg = 0.020 KB total ukey num = 0.008356385 Billion total entry num = 0.008356385 Billion write speed all = 0.019205682 MB/sec (with seq num) write speed all = 0.018190865 MB/sec (without seq num) 2024/11/23-03:10:18.125684 140025171461696 [compaction_job.cc:1948] [default] [JOB 18051] Generated table #242112: 923080 keys, 37022074 bytes, temperature: kWarm 2024/11/23-03:10:18.125746 140025171461696 EVENT_LOG_v1 {"time_micros": 1732302618125715, "cf_name": "default", "job": 18051, "event": "table_file_creation", "file_number": 242112, "file_size": 37022074, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 3751819057, "table_properties": {"data_size": 30831264, "index_size": 1756096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25846240, "raw_average_key_size": 28, "raw_value_size": 108371628, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923080, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732302614, "oldest_key_time": 0, "file_creation_time": 1732302614, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 242112, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:10:18.161855 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242113.sst) /dev/shm/ToplingTemp/Topling-390667-5CmTuA 2024/11/23-03:10:18.974656 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242113.sst): first pass time = 0.81's, 165.138'MB/sec 2024/11/23-03:10:20.478032 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242113.sst): old prealloc_size = 147639500, real_size = 33355248 2024/11/23-03:10:20.478566 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242113.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926921 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.80 avg-zval = 33.82 2024/11/23-03:10:20.479511 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242113.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:10:21.440790 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242113.sst): second pass time = 1.50's, 72.075'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 39.024'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3482.750'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.278'MB/sec write SST data time = 0.13's, 291.910'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 25696.993'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 72.075'MB/sec zip pipeline throughput = 105.953'MB/sec entries = 926921 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.80 avg-zval = 33.82 usrkeys = 926921 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.80 avg-zval = 33.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926921 bytes = 3360096 } seq expand size = 6488447 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.4432 value = 3.4536 dict = 1.00 all = 3.5725 } Zip/UnZip{ index = 0.0958 value = 0.2896 dict = 1.00 all = 0.2799 } ---------------------------- total value len = 1.145948 GB avg = 0.123 KB (by entry num) total key len = 0.259933 GB avg = 0.028 KB total ukey len = 0.185666 GB avg = 0.020 KB total ukey num = 0.009283306 Billion total entry num = 0.009283306 Billion write speed all = 0.021340554 MB/sec (with seq num) write speed all = 0.020213227 MB/sec (without seq num) 2024/11/23-03:10:21.477653 140025171461696 [compaction_job.cc:1948] [default] [JOB 18051] Generated table #242113: 926921 keys, 37569456 bytes, temperature: kWarm 2024/11/23-03:10:21.477713 140025171461696 EVENT_LOG_v1 {"time_micros": 1732302621477682, "cf_name": "default", "job": 18051, "event": "table_file_creation", "file_number": 242113, "file_size": 37569456, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 3751820099, "table_properties": {"data_size": 31348192, "index_size": 1775168, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25953788, "raw_average_key_size": 28, "raw_value_size": 108264006, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926921, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732302618, "oldest_key_time": 0, "file_creation_time": 1732302618, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 242113, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:10:21.515267 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242114.sst) /dev/shm/ToplingTemp/Topling-390667-O3p0UA 2024/11/23-03:10:21.857789 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242114.sst): first pass time = 0.34's, 189.903'MB/sec 2024/11/23-03:10:22.505530 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242114.sst): old prealloc_size = 147639500, real_size = 16062768 2024/11/23-03:10:22.505889 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242114.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 453442 prefix = 4 raw-key = 0.0073 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0523 GB zip-val = 0.0151 GB avg-val = 115.43 avg-zval = 33.24 2024/11/23-03:10:22.506264 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242114.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.065 GB 2024/11/23-03:10:23.033269 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00/242114.sst): second pass time = 0.65's, 80.896'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.53's, 34.021'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2619.520'MB/sec (index lex order gen) rebuild zvType time = 0.42's, 0.272'MB/sec write SST data time = 0.11's, 161.375'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.527'MB, throughput = 18447.236'MB/sec zip my value time = 0.65's, unzip length = 0.052'GB zip my value throughput = 80.896'MB/sec zip pipeline throughput = 80.896'MB/sec entries = 453442 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.43 avg-zval = 33.24 usrkeys = 453442 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.43 avg-zval = 33.24 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 453442 bytes = 1473696 } seq expand size = 3174094 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0091 GB value = 0.0523 GB dict = 0.53 MB all = 0.0650 GB } __ZipSize{ index = 0.0009 GB value = 0.0151 GB dict = 0.53 MB all = 0.0180 GB } UnZip/Zip{ index = 10.3673 value = 3.4723 dict = 1.00 all = 3.6228 } Zip/UnZip{ index = 0.0965 value = 0.2880 dict = 1.00 all = 0.2760 } ---------------------------- total value len = 1.201465 GB avg = 0.123 KB (by entry num) total key len = 0.272629 GB avg = 0.028 KB total ukey len = 0.194735 GB avg = 0.020 KB total ukey num = 0.009736748 Billion total entry num = 0.009736748 Billion write speed all = 0.022375456 MB/sec (with seq num) write speed all = 0.021193093 MB/sec (without seq num) 2024/11/23-03:10:23.054995 140025171461696 [compaction_job.cc:1948] [default] [JOB 18051] Generated table #242114: 453442 keys, 17952901 bytes, temperature: kWarm 2024/11/23-03:10:23.055079 140025171461696 EVENT_LOG_v1 {"time_micros": 1732302623055034, "cf_name": "default", "job": 18051, "event": "table_file_creation", "file_number": 242114, "file_size": 17952901, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3700714014, "largest_seqno": 3751818897, "table_properties": {"data_size": 15074464, "index_size": 874752, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 12696376, "raw_average_key_size": 28, "raw_value_size": 52342768, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 453442, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732302621, "oldest_key_time": 0, "file_creation_time": 1732302621, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 242114, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:10:23.077792 140025171461696 [compaction_job.cc:703] job-18051: subcompact[0], size: 0.393969 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18051/att-00 [242104,242105,242106,242107,242108,242109,242110,242111,242112,242113,242114] 2024/11/23-03:10:23.208170 140025171461696 (Original Log Time 2024/11/23-03:09:44.183402) EVENT_LOG_v1 {"time_micros": 1732302584183341, "job": 18051, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [242098], "files_L3": [242019, 242022, 242023, 242024, 242025, 242039, 242040, 242041, 242042, 242043, 242044], "score": 0.999252, "input_data_size": 412343883, "oldest_snapshot_seqno": 3751878795} 2024/11/23-03:10:23.208176 140025171461696 (Original Log Time 2024/11/23-03:10:23.207127) [compaction_job.cc:2022] [default] [JOB 18051] Compacted 1@2 + 11@3 files to L3 => 393968571 bytes 2024/11/23-03:10:23.208178 140025171461696 (Original Log Time 2024/11/23-03:10:23.208087) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 11 0 0 0] max score 0.01, MB/sec: 10.6 rd, 10.1 wr, level 3, files in(1, 11) out(11 +0 blob) MB in(25.7, 367.6 +0.0 blob) out(375.7 +0.0 blob), read-write-amplify(29.9) write-amplify(14.6) OK, records in: 9791925, records dropped: 55177 output_compression: Snappy 2024/11/23-03:10:23.208188 140025171461696 (Original Log Time 2024/11/23-03:10:23.208146) EVENT_LOG_v1 {"time_micros": 1732302623208119, "cf": "default", "job": 18051, "event": "compaction_finished", "compaction_time_micros": 38894403, "compaction_time_cpu_micros": 16886332, "output_level": 3, "num_output_files": 11, "total_output_size": 393968571, "num_input_records": 9791925, "num_output_records": 9736748, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 247592036, "file_range_sync_nanos": 0, "file_fsync_nanos": 11153, "file_prepare_write_nanos": 26984, "lsm_state": [0, 0, 0, 11, 0, 0, 0]} 2024/11/23-03:10:23.341663 140025171461696 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732302583,"executesMs":39180,"compactionJobId":18051,"attempt":0,"compactionInputRawBytes":1415052514,"compactionInputZipBytes":412343883,"compactionOutputRawBytes":0,"compactionOutputZipBytes":393968571,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-c76ae1454813e21cfd5d4ebd62736d90-ec76cd8cf31b4c6d-00"}}, response = 2024/11/23-03:10:23.341731 140025171461696 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-18051/att-00: olev 3, work 39.180 s, result 11.052 ms, install 1.088 ms, input{raw 1.318 GiB zip 393.242 MiB}