2024/11/23-07:12:06.869785 140025020393024 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00: fork to child time = 0.009259 sec 2024/11/23-07:12:06.870471 140025020393024 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00 2024/11/23-07:12:06.878950 140025020393024 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00 2024/11/23-07:12:06.880066 140025020393024 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-07:12:06.880327 140025020393024 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-07:12:06.929039 140025020393024 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-21644/att-00/MANIFEST-206347 2024/11/23-07:12:06.957640 140025020393024 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-21644/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 249984, last_sequence is 3853871931, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 249828 2024/11/23-07:12:06.957650 140025020393024 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-07:12:06.957756 140025020393024 [version_set.cc:5721] Creating manifest 249984 2024/11/23-07:12:06.987413 140025020393024 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-07:12:07.007079 140025020393024 [compaction_job.cc:2437] [default] [JOB 21644] Compacting 1@2 + 14@3 files to L3, score 0.98, subcompactions 1 : 1 2024/11/23-07:12:07.007099 140025020393024 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 16226 Base level 2, inputs: [249973(27MB)], [249891(36MB) 249892(35MB) 249893(35MB) 249894(35MB) 249895(33MB) 249896(35MB) 249897(34MB) 249898(36MB) 249899(36MB) 249900(35MB) 249901(34MB) 249933(34MB) 249934(34MB) 249935(34MB)] 2024/11/23-07:12:07.012850 140025020393024 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-07:12:07.012882 140025020393024 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-07:12:07.012902 140025020393024 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-07:12:07.012908 140025020393024 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-07:12:07.012918 140025020393024 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-07:12:07.483618 140025020393024 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249985.sst) /dev/shm/ToplingTemp/Topling-444076-6hkB98 2024/11/23-07:12:08.634934 140025020393024 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249985.sst): first pass time = 1.15's, 116.583'MB/sec 2024/11/23-07:12:12.203966 140025020393024 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249985.sst): old prealloc_size = 147639500, real_size = 34251264 2024/11/23-07:12:12.204580 140025020393024 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249985.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923835 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0323 GB avg-val = 117.28 avg-zval = 34.91 2024/11/23-07:12:12.205450 140025020393024 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249985.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:12:13.806091 140025020393024 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249985.sst): second pass time = 3.57's, 30.367'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.60's, 23.932'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2979.316'MB/sec (index lex order gen) rebuild zvType time = 1.29's, 0.179'MB/sec write SST data time = 0.31's, 123.204'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 35941.481'MB/sec zip my value time = 3.57's, unzip length = 0.108'GB zip my value throughput = 30.367'MB/sec zip pipeline throughput = 111.820'MB/sec entries = 923835 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.28 avg-zval = 34.91 usrkeys = 923835 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.28 avg-zval = 34.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923835 bytes = 3233440 } seq expand size = 6466845 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0323 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.4434 value = 3.3596 dict = 1.00 all = 3.5006 } Zip/UnZip{ index = 0.0958 value = 0.2977 dict = 1.00 all = 0.2857 } ---------------------------- total value len = 0.114817 GB avg = 0.124 KB (by entry num) total key len = 0.025867 GB avg = 0.028 KB total ukey len = 0.018477 GB avg = 0.020 KB total ukey num = 0.000923835 Billion total entry num = 0.000923835 Billion write speed all = 0.001750010 MB/sec (with seq num) write speed all = 0.001658076 MB/sec (without seq num) 2024/11/23-07:12:13.847403 140025020393024 [compaction_job.cc:1948] [default] [JOB 21644] Generated table #249985: 923835 keys, 38341798 bytes, temperature: kWarm 2024/11/23-07:12:13.847579 140025020393024 EVENT_LOG_v1 {"time_micros": 1732317133847483, "cf_name": "default", "job": 21644, "event": "table_file_creation", "file_number": 249985, "file_size": 38341798, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3661838825, "largest_seqno": 3853849032, "table_properties": {"data_size": 32250928, "index_size": 1769216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25867380, "raw_average_key_size": 28, "raw_value_size": 108350350, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923835, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732317127, "oldest_key_time": 0, "file_creation_time": 1732317127, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249985, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:12:13.925520 140025020393024 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249986.sst) /dev/shm/ToplingTemp/Topling-444076-qYo8f9 2024/11/23-07:12:15.401500 140025020393024 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249986.sst): first pass time = 1.48's, 90.937'MB/sec 2024/11/23-07:12:18.572492 140025020393024 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249986.sst): old prealloc_size = 147639500, real_size = 33222912 2024/11/23-07:12:18.573210 140025020393024 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249986.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928487 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.56 avg-zval = 33.63 2024/11/23-07:12:18.574013 140025020393024 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249986.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:12:20.027817 140025020393024 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249986.sst): second pass time = 3.17's, 34.137'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.46's, 25.569'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2528.090'MB/sec (index lex order gen) rebuild zvType time = 1.30's, 0.179'MB/sec write SST data time = 0.15's, 241.285'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32199.553'MB/sec zip my value time = 3.17's, unzip length = 0.108'GB zip my value throughput = 34.137'MB/sec zip pipeline throughput = 93.933'MB/sec entries = 928487 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.56 avg-zval = 33.63 usrkeys = 928487 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.56 avg-zval = 33.63 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928487 bytes = 3133664 } seq expand size = 6499409 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5120 value = 3.4659 dict = 1.00 all = 3.6070 } Zip/UnZip{ index = 0.0951 value = 0.2885 dict = 1.00 all = 0.2772 } ---------------------------- total value len = 0.229537 GB avg = 0.124 KB (by entry num) total key len = 0.051865 GB avg = 0.028 KB total ukey len = 0.037046 GB avg = 0.020 KB total ukey num = 0.001852322 Billion total entry num = 0.001852322 Billion write speed all = 0.003500155 MB/sec (with seq num) write speed all = 0.003315838 MB/sec (without seq num) 2024/11/23-07:12:20.065474 140025020393024 [compaction_job.cc:1948] [default] [JOB 21644] Generated table #249986: 928487 keys, 37210068 bytes, temperature: kWarm 2024/11/23-07:12:20.065537 140025020393024 EVENT_LOG_v1 {"time_micros": 1732317140065504, "cf_name": "default", "job": 21644, "event": "table_file_creation", "file_number": 249986, "file_size": 37210068, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3748831343, "largest_seqno": 3853849043, "table_properties": {"data_size": 31224048, "index_size": 1766528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25997636, "raw_average_key_size": 28, "raw_value_size": 108220122, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928487, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732317133, "oldest_key_time": 0, "file_creation_time": 1732317133, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249986, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:12:20.101784 140025020393024 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249987.sst) /dev/shm/ToplingTemp/Topling-444076-mUPJa9 2024/11/23-07:12:21.211950 140025020393024 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249987.sst): first pass time = 1.11's, 120.903'MB/sec 2024/11/23-07:12:23.040672 140025020393024 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249987.sst): old prealloc_size = 147639500, real_size = 32596576 2024/11/23-07:12:23.041268 140025020393024 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249987.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 935360 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0306 GB avg-val = 115.49 avg-zval = 32.70 2024/11/23-07:12:23.042062 140025020393024 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249987.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:12:23.993138 140025020393024 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249987.sst): second pass time = 1.83's, 59.118'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 38.676'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3094.462'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.285'MB/sec write SST data time = 0.13's, 282.975'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33183.721'MB/sec zip my value time = 1.83's, unzip length = 0.108'GB zip my value throughput = 59.118'MB/sec zip pipeline throughput = 116.859'MB/sec entries = 935360 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.49 avg-zval = 32.70 usrkeys = 935360 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.49 avg-zval = 32.70 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 935360 bytes = 3390688 } seq expand size = 6547520 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5114 value = 3.5323 dict = 1.00 all = 3.6435 } Zip/UnZip{ index = 0.0951 value = 0.2831 dict = 1.00 all = 0.2745 } ---------------------------- total value len = 0.344112 GB avg = 0.123 KB (by entry num) total key len = 0.078055 GB avg = 0.028 KB total ukey len = 0.055754 GB avg = 0.020 KB total ukey num = 0.002787682 Billion total entry num = 0.002787682 Billion write speed all = 0.005250776 MB/sec (with seq num) write speed all = 0.004973398 MB/sec (without seq num) 2024/11/23-07:12:24.028689 140025020393024 [compaction_job.cc:1948] [default] [JOB 21644] Generated table #249987: 935360 keys, 36837252 bytes, temperature: kWarm 2024/11/23-07:12:24.028751 140025020393024 EVENT_LOG_v1 {"time_micros": 1732317144028719, "cf_name": "default", "job": 21644, "event": "table_file_creation", "file_number": 249987, "file_size": 36837252, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038075, "largest_seqno": 3853849207, "table_properties": {"data_size": 30582864, "index_size": 1779712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26190080, "raw_average_key_size": 28, "raw_value_size": 108027788, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 935360, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732317140, "oldest_key_time": 0, "file_creation_time": 1732317140, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249987, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:12:24.062926 140025020393024 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249988.sst) /dev/shm/ToplingTemp/Topling-444076-zlVIZ7 2024/11/23-07:12:24.908669 140025020393024 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249988.sst): first pass time = 0.85's, 158.705'MB/sec 2024/11/23-07:12:26.371086 140025020393024 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249988.sst): old prealloc_size = 147639500, real_size = 32321968 2024/11/23-07:12:26.371619 140025020393024 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249988.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928155 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0303 GB avg-val = 116.61 avg-zval = 32.66 2024/11/23-07:12:26.372427 140025020393024 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249988.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:12:27.275840 140025020393024 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249988.sst): second pass time = 1.46's, 74.040'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 40.387'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3442.248'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.299'MB/sec write SST data time = 0.13's, 283.061'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 38129.308'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 74.040'MB/sec zip pipeline throughput = 74.040'MB/sec entries = 928155 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.61 avg-zval = 32.66 usrkeys = 928155 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.61 avg-zval = 32.66 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928155 bytes = 3364576 } seq expand size = 6497085 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.4458 value = 3.5704 dict = 1.00 all = 3.6732 } Zip/UnZip{ index = 0.0957 value = 0.2801 dict = 1.00 all = 0.2722 } ---------------------------- total value len = 0.458839 GB avg = 0.123 KB (by entry num) total key len = 0.104043 GB avg = 0.028 KB total ukey len = 0.074317 GB avg = 0.020 KB total ukey num = 0.003715837 Billion total entry num = 0.003715837 Billion write speed all = 0.007000657 MB/sec (with seq num) write speed all = 0.006630941 MB/sec (without seq num) 2024/11/23-07:12:27.315400 140025020393024 [compaction_job.cc:1948] [default] [JOB 21644] Generated table #249988: 928155 keys, 36540190 bytes, temperature: kWarm 2024/11/23-07:12:27.315459 140025020393024 EVENT_LOG_v1 {"time_micros": 1732317147315430, "cf_name": "default", "job": 21644, "event": "table_file_creation", "file_number": 249988, "file_size": 36540190, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038106, "largest_seqno": 3853849091, "table_properties": {"data_size": 30312672, "index_size": 1777088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25988340, "raw_average_key_size": 28, "raw_value_size": 108229494, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928155, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732317144, "oldest_key_time": 0, "file_creation_time": 1732317144, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249988, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:12:27.351121 140025020393024 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249989.sst) /dev/shm/ToplingTemp/Topling-444076-ly1t0a 2024/11/23-07:12:28.188322 140025020393024 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249989.sst): first pass time = 0.84's, 160.324'MB/sec 2024/11/23-07:12:29.609126 140025020393024 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249989.sst): old prealloc_size = 147639500, real_size = 31044720 2024/11/23-07:12:29.609638 140025020393024 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249989.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922774 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0290 GB avg-val = 117.45 avg-zval = 31.48 2024/11/23-07:12:29.610419 140025020393024 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249989.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:12:30.453702 140025020393024 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249989.sst): second pass time = 1.42's, 76.328'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 41.597'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3608.387'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 277.274'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 35644.221'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 76.328'MB/sec zip pipeline throughput = 76.328'MB/sec entries = 922774 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.45 avg-zval = 31.48 usrkeys = 922774 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.45 avg-zval = 31.48 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922774 bytes = 3229728 } seq expand size = 6459418 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0290 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.4435 value = 3.7312 dict = 1.00 all = 3.8205 } Zip/UnZip{ index = 0.0958 value = 0.2680 dict = 1.00 all = 0.2617 } ---------------------------- total value len = 0.573678 GB avg = 0.124 KB (by entry num) total key len = 0.129881 GB avg = 0.028 KB total ukey len = 0.092772 GB avg = 0.020 KB total ukey num = 0.004638611 Billion total entry num = 0.004638611 Billion write speed all = 0.008749937 MB/sec (with seq num) write speed all = 0.008288426 MB/sec (without seq num) 2024/11/23-07:12:30.489592 140025020393024 [compaction_job.cc:1948] [default] [JOB 21644] Generated table #249989: 922774 keys, 35130774 bytes, temperature: kWarm 2024/11/23-07:12:30.489655 140025020393024 EVENT_LOG_v1 {"time_micros": 1732317150489622, "cf_name": "default", "job": 21644, "event": "table_file_creation", "file_number": 249989, "file_size": 35130774, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3673151148, "largest_seqno": 3853849105, "table_properties": {"data_size": 29046688, "index_size": 1767168, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25837672, "raw_average_key_size": 28, "raw_value_size": 108380176, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922774, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732317147, "oldest_key_time": 0, "file_creation_time": 1732317147, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249989, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:12:30.524862 140025020393024 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249990.sst) /dev/shm/ToplingTemp/Topling-444076-4BpOJa 2024/11/23-07:12:31.414800 140025020393024 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249990.sst): first pass time = 0.89's, 150.823'MB/sec 2024/11/23-07:12:32.824897 140025020393024 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249990.sst): old prealloc_size = 147639500, real_size = 32888928 2024/11/23-07:12:32.825444 140025020393024 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249990.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930866 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.19 avg-zval = 33.17 2024/11/23-07:12:32.826242 140025020393024 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249990.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:12:33.742038 140025020393024 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249990.sst): second pass time = 1.41's, 76.758'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.469'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3386.946'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.295'MB/sec write SST data time = 0.13's, 291.155'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35275.802'MB/sec zip my value time = 1.41's, unzip length = 0.108'GB zip my value throughput = 76.758'MB/sec zip pipeline throughput = 76.758'MB/sec entries = 930866 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.19 avg-zval = 33.17 usrkeys = 930866 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.19 avg-zval = 33.17 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930866 bytes = 3374400 } seq expand size = 6516062 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.4458 value = 3.5031 dict = 1.00 all = 3.6163 } Zip/UnZip{ index = 0.0957 value = 0.2855 dict = 1.00 all = 0.2765 } ---------------------------- total value len = 0.688348 GB avg = 0.124 KB (by entry num) total key len = 0.155945 GB avg = 0.028 KB total ukey len = 0.111390 GB avg = 0.020 KB total ukey num = 0.005569477 Billion total entry num = 0.005569477 Billion write speed all = 0.010499769 MB/sec (with seq num) write speed all = 0.009945665 MB/sec (without seq num) 2024/11/23-07:12:33.786497 140025020393024 [compaction_job.cc:1948] [default] [JOB 21644] Generated table #249990: 930866 keys, 37114778 bytes, temperature: kWarm 2024/11/23-07:12:33.786557 140025020393024 EVENT_LOG_v1 {"time_micros": 1732317153786527, "cf_name": "default", "job": 21644, "event": "table_file_creation", "file_number": 249990, "file_size": 37114778, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038136, "largest_seqno": 3853849118, "table_properties": {"data_size": 30873744, "index_size": 1782272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26064248, "raw_average_key_size": 28, "raw_value_size": 108153556, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930866, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732317150, "oldest_key_time": 0, "file_creation_time": 1732317150, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249990, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:12:33.823583 140025020393024 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249991.sst) /dev/shm/ToplingTemp/Topling-444076-ZdxAL9 2024/11/23-07:12:34.676673 140025020393024 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249991.sst): first pass time = 0.85's, 157.339'MB/sec 2024/11/23-07:12:36.093912 140025020393024 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249991.sst): old prealloc_size = 147639500, real_size = 33388560 2024/11/23-07:12:36.094447 140025020393024 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249991.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925774 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.98 avg-zval = 33.91 2024/11/23-07:12:36.095223 140025020393024 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249991.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:12:37.025801 140025020393024 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249991.sst): second pass time = 1.42's, 76.471'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.224'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3503.155'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.287'MB/sec write SST data time = 0.13's, 296.984'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36571.477'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 76.471'MB/sec zip pipeline throughput = 76.471'MB/sec entries = 925774 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.98 avg-zval = 33.91 usrkeys = 925774 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.98 avg-zval = 33.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925774 bytes = 3240224 } seq expand size = 6480418 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5106 value = 3.4494 dict = 1.00 all = 3.5807 } Zip/UnZip{ index = 0.0951 value = 0.2899 dict = 1.00 all = 0.2793 } ---------------------------- total value len = 0.803124 GB avg = 0.124 KB (by entry num) total key len = 0.181867 GB avg = 0.028 KB total ukey len = 0.129905 GB avg = 0.020 KB total ukey num = 0.006495251 Billion total entry num = 0.006495251 Billion write speed all = 0.012249015 MB/sec (with seq num) write speed all = 0.011602833 MB/sec (without seq num) 2024/11/23-07:12:37.065020 140025020393024 [compaction_job.cc:1948] [default] [JOB 21644] Generated table #249991: 925774 keys, 37483510 bytes, temperature: kWarm 2024/11/23-07:12:37.065082 140025020393024 EVENT_LOG_v1 {"time_micros": 1732317157065051, "cf_name": "default", "job": 21644, "event": "table_file_creation", "file_number": 249991, "file_size": 37483510, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3694505702, "largest_seqno": 3853849149, "table_properties": {"data_size": 31395328, "index_size": 1761600, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25921672, "raw_average_key_size": 28, "raw_value_size": 108296152, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925774, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732317153, "oldest_key_time": 0, "file_creation_time": 1732317153, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249991, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:12:37.099212 140025020393024 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249992.sst) /dev/shm/ToplingTemp/Topling-444076-F7Ds1a 2024/11/23-07:12:37.928306 140025020393024 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249992.sst): first pass time = 0.83's, 161.892'MB/sec 2024/11/23-07:12:39.580979 140025020393024 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249992.sst): old prealloc_size = 147639500, real_size = 36773648 2024/11/23-07:12:39.581564 140025020393024 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249992.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929191 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0348 GB avg-val = 116.45 avg-zval = 37.40 2024/11/23-07:12:39.581868 140025020393024 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249992.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:12:40.270670 140025020393024 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249992.sst): second pass time = 1.65's, 65.499'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.69's, 55.685'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3176.543'MB/sec (index lex order gen) rebuild zvType time = 0.55's, 0.423'MB/sec write SST data time = 0.14's, 273.431'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34882.747'MB/sec zip my value time = 1.65's, unzip length = 0.108'GB zip my value throughput = 65.499'MB/sec zip pipeline throughput = 65.499'MB/sec entries = 929191 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.45 avg-zval = 37.40 usrkeys = 929191 avg-key = 20.00 avg-zkey = 1.93 avg-val = 117.71 avg-zval = 37.40 TagRS{ kinds = RS_Key0_TagN bytes = 145280 } TagArray{ size = 167971 bytes = 629904 } seq expand size = 1175797 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0348 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.3760 value = 3.1137 dict = 1.00 all = 3.4948 } Zip/UnZip{ index = 0.0964 value = 0.3212 dict = 1.00 all = 0.2861 } ---------------------------- total value len = 0.912501 GB avg = 0.123 KB (by entry num) total key len = 0.207884 GB avg = 0.028 KB total ukey len = 0.148489 GB avg = 0.020 KB total ukey num = 0.007424442 Billion total entry num = 0.007424442 Billion write speed all = 0.013932161 MB/sec (with seq num) write speed all = 0.013193568 MB/sec (without seq num) 2024/11/23-07:12:40.307565 140025020393024 [compaction_job.cc:1948] [default] [JOB 21644] Generated table #249992: 929191 keys, 38405123 bytes, temperature: kWarm 2024/11/23-07:12:40.307627 140025020393024 EVENT_LOG_v1 {"time_micros": 1732317160307595, "cf_name": "default", "job": 21644, "event": "table_file_creation", "file_number": 249992, "file_size": 38405123, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3853846765, "table_properties": {"data_size": 34750144, "index_size": 1791040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26017348, "raw_average_key_size": 28, "raw_value_size": 108200458, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929191, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732317157, "oldest_key_time": 0, "file_creation_time": 1732317157, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249992, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:12:40.342820 140025020393024 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249993.sst) /dev/shm/ToplingTemp/Topling-444076-82Pon8 2024/11/23-07:12:41.092933 140025020393024 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249993.sst): first pass time = 0.75's, 178.939'MB/sec 2024/11/23-07:12:42.766757 140025020393024 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249993.sst): old prealloc_size = 147639500, real_size = 36782368 2024/11/23-07:12:42.767307 140025020393024 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249993.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926474 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0348 GB avg-val = 116.87 avg-zval = 37.55 2024/11/23-07:12:42.767469 140025020393024 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249993.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:12:43.410624 140025020393024 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249993.sst): second pass time = 1.67's, 64.726'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 58.958'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3377.231'MB/sec (index lex order gen) rebuild zvType time = 0.52's, 0.446'MB/sec write SST data time = 0.12's, 306.738'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33240.161'MB/sec zip my value time = 1.67's, unzip length = 0.108'GB zip my value throughput = 64.726'MB/sec zip pipeline throughput = 67.126'MB/sec entries = 926474 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.87 avg-zval = 37.55 usrkeys = 926474 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.31 avg-zval = 37.55 TagRS{ kinds = RS_Key0_TagN bytes = 144880 } TagArray{ size = 58203 bytes = 174624 } seq expand size = 407421 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0348 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.5128 value = 3.1125 dict = 1.00 all = 3.5358 } Zip/UnZip{ index = 0.0951 value = 0.3213 dict = 1.00 all = 0.2828 } ---------------------------- total value len = 1.021185 GB avg = 0.122 KB (by entry num) total key len = 0.233826 GB avg = 0.028 KB total ukey len = 0.167018 GB avg = 0.020 KB total ukey num = 0.008350916 Billion total entry num = 0.008350916 Billion write speed all = 0.015605638 MB/sec (with seq num) write speed all = 0.014774911 MB/sec (without seq num) 2024/11/23-07:12:43.447942 140025020393024 [compaction_job.cc:1948] [default] [JOB 21644] Generated table #249993: 926474 keys, 37960095 bytes, temperature: kWarm 2024/11/23-07:12:43.448005 140025020393024 EVENT_LOG_v1 {"time_micros": 1732317163447973, "cf_name": "default", "job": 21644, "event": "table_file_creation", "file_number": 249993, "file_size": 37960095, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3285641070, "table_properties": {"data_size": 34787984, "index_size": 1762560, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25941272, "raw_average_key_size": 28, "raw_value_size": 108276564, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926474, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732317160, "oldest_key_time": 0, "file_creation_time": 1732317160, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249993, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:12:43.484219 140025020393024 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249994.sst) /dev/shm/ToplingTemp/Topling-444076-VdTFab 2024/11/23-07:12:44.305907 140025020393024 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249994.sst): first pass time = 0.82's, 163.352'MB/sec 2024/11/23-07:12:45.877406 140025020393024 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249994.sst): old prealloc_size = 147639500, real_size = 33084096 2024/11/23-07:12:45.877982 140025020393024 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249994.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927738 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.67 avg-zval = 33.50 2024/11/23-07:12:45.880572 140025020393024 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249994.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:12:46.806446 140025020393024 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249994.sst): second pass time = 1.57's, 68.908'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.276'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3226.899'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.290'MB/sec write SST data time = 0.13's, 290.869'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31010.965'MB/sec zip my value time = 1.57's, unzip length = 0.108'GB zip my value throughput = 68.908'MB/sec zip pipeline throughput = 120.092'MB/sec entries = 927738 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.67 avg-zval = 33.50 usrkeys = 927738 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.67 avg-zval = 33.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927738 bytes = 3479040 } seq expand size = 6494166 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4415 value = 3.4832 dict = 1.00 all = 3.5871 } Zip/UnZip{ index = 0.0958 value = 0.2871 dict = 1.00 all = 0.2788 } ---------------------------- total value len = 1.135920 GB avg = 0.122 KB (by entry num) total key len = 0.259802 GB avg = 0.028 KB total ukey len = 0.185573 GB avg = 0.020 KB total ukey num = 0.009278654 Billion total entry num = 0.009278654 Billion write speed all = 0.017354612 MB/sec (with seq num) write speed all = 0.016431635 MB/sec (without seq num) 2024/11/23-07:12:46.843864 140025020393024 [compaction_job.cc:1948] [default] [JOB 21644] Generated table #249994: 927738 keys, 37417272 bytes, temperature: kWarm 2024/11/23-07:12:46.843928 140025020393024 EVENT_LOG_v1 {"time_micros": 1732317166843897, "cf_name": "default", "job": 21644, "event": "table_file_creation", "file_number": 249994, "file_size": 37417272, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3285645833, "largest_seqno": 3853849451, "table_properties": {"data_size": 31074928, "index_size": 1777024, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25976664, "raw_average_key_size": 28, "raw_value_size": 108241180, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927738, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732317163, "oldest_key_time": 0, "file_creation_time": 1732317163, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249994, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:12:46.878934 140025020393024 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249995.sst) /dev/shm/ToplingTemp/Topling-444076-vHO368 2024/11/23-07:12:47.784007 140025020393024 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249995.sst): first pass time = 0.91's, 148.301'MB/sec 2024/11/23-07:12:49.160270 140025020393024 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249995.sst): old prealloc_size = 147639500, real_size = 32588416 2024/11/23-07:12:49.160867 140025020393024 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249995.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929975 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0306 GB avg-val = 116.32 avg-zval = 32.89 2024/11/23-07:12:49.163429 140025020393024 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249995.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:12:50.085395 140025020393024 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249995.sst): second pass time = 1.38's, 78.663'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 39.667'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3074.048'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.291'MB/sec write SST data time = 0.13's, 289.196'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34768.547'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.663'MB/sec zip pipeline throughput = 78.663'MB/sec entries = 929975 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.32 avg-zval = 32.89 usrkeys = 929975 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.32 avg-zval = 32.89 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929975 bytes = 3254928 } seq expand size = 6509825 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5128 value = 3.5368 dict = 1.00 all = 3.6576 } Zip/UnZip{ index = 0.0951 value = 0.2827 dict = 1.00 all = 0.2734 } ---------------------------- total value len = 1.250608 GB avg = 0.123 KB (by entry num) total key len = 0.285842 GB avg = 0.028 KB total ukey len = 0.204173 GB avg = 0.020 KB total ukey num = 0.010208629 Billion total entry num = 0.010208629 Billion write speed all = 0.019103661 MB/sec (with seq num) write speed all = 0.018088217 MB/sec (without seq num) 2024/11/23-07:12:50.126581 140025020393024 [compaction_job.cc:1948] [default] [JOB 21644] Generated table #249995: 929975 keys, 36696040 bytes, temperature: kWarm 2024/11/23-07:12:50.126643 140025020393024 EVENT_LOG_v1 {"time_micros": 1732317170126612, "cf_name": "default", "job": 21644, "event": "table_file_creation", "file_number": 249995, "file_size": 36696040, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3631595348, "largest_seqno": 3853849483, "table_properties": {"data_size": 30586544, "index_size": 1769216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26039300, "raw_average_key_size": 28, "raw_value_size": 108178482, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929975, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732317166, "oldest_key_time": 0, "file_creation_time": 1732317166, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249995, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:12:50.165649 140025020393024 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249996.sst) /dev/shm/ToplingTemp/Topling-444076-CMY5Za 2024/11/23-07:12:51.061436 140025020393024 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249996.sst): first pass time = 0.90's, 149.839'MB/sec 2024/11/23-07:12:52.403037 140025020393024 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249996.sst): old prealloc_size = 147639500, real_size = 32920560 2024/11/23-07:12:52.403537 140025020393024 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249996.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928678 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.53 avg-zval = 33.30 2024/11/23-07:12:52.404308 140025020393024 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249996.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:12:53.354454 140025020393024 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249996.sst): second pass time = 1.34's, 80.702'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 38.793'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3681.067'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.282'MB/sec write SST data time = 0.13's, 291.042'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31964.589'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.702'MB/sec zip pipeline throughput = 80.702'MB/sec entries = 928678 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.53 avg-zval = 33.30 usrkeys = 928678 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.53 avg-zval = 33.30 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928678 bytes = 3134304 } seq expand size = 6500746 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5138 value = 3.4996 dict = 1.00 all = 3.6365 } Zip/UnZip{ index = 0.0951 value = 0.2857 dict = 1.00 all = 0.2750 } ---------------------------- total value len = 1.365324 GB avg = 0.123 KB (by entry num) total key len = 0.311845 GB avg = 0.028 KB total ukey len = 0.222746 GB avg = 0.020 KB total ukey num = 0.011137307 Billion total entry num = 0.011137307 Billion write speed all = 0.020852457 MB/sec (with seq num) write speed all = 0.019744684 MB/sec (without seq num) 2024/11/23-07:12:53.390711 140025020393024 [compaction_job.cc:1948] [default] [JOB 21644] Generated table #249996: 928678 keys, 36908164 bytes, temperature: kWarm 2024/11/23-07:12:53.390775 140025020393024 EVENT_LOG_v1 {"time_micros": 1732317173390744, "cf_name": "default", "job": 21644, "event": "table_file_creation", "file_number": 249996, "file_size": 36908164, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3736072251, "largest_seqno": 3853849493, "table_properties": {"data_size": 30921632, "index_size": 1766592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26002984, "raw_average_key_size": 28, "raw_value_size": 108214840, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928678, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732317170, "oldest_key_time": 0, "file_creation_time": 1732317170, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249996, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:12:53.425078 140025020393024 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249997.sst) /dev/shm/ToplingTemp/Topling-444076-coqbDa 2024/11/23-07:12:54.222185 140025020393024 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249997.sst): first pass time = 0.80's, 168.390'MB/sec 2024/11/23-07:12:55.551602 140025020393024 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249997.sst): old prealloc_size = 147639500, real_size = 32966448 2024/11/23-07:12:55.552141 140025020393024 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249997.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929006 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.47 avg-zval = 33.32 2024/11/23-07:12:55.554588 140025020393024 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249997.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:12:56.510745 140025020393024 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249997.sst): second pass time = 1.33's, 81.434'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 38.771'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3421.746'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.280'MB/sec write SST data time = 0.13's, 287.505'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 28904.465'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.434'MB/sec zip pipeline throughput = 81.434'MB/sec entries = 929006 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.47 avg-zval = 33.32 usrkeys = 929006 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.47 avg-zval = 33.32 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929006 bytes = 3367664 } seq expand size = 6503042 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.4437 value = 3.4956 dict = 1.00 all = 3.6093 } Zip/UnZip{ index = 0.0958 value = 0.2861 dict = 1.00 all = 0.2771 } ---------------------------- total value len = 1.480033 GB avg = 0.123 KB (by entry num) total key len = 0.337857 GB avg = 0.028 KB total ukey len = 0.241326 GB avg = 0.020 KB total ukey num = 0.012066313 Billion total entry num = 0.012066313 Billion write speed all = 0.022601171 MB/sec (with seq num) write speed all = 0.021401041 MB/sec (without seq num) 2024/11/23-07:12:56.554846 140025020393024 [compaction_job.cc:1948] [default] [JOB 21644] Generated table #249997: 929006 keys, 37186806 bytes, temperature: kWarm 2024/11/23-07:12:56.554910 140025020393024 EVENT_LOG_v1 {"time_micros": 1732317176554879, "cf_name": "default", "job": 21644, "event": "table_file_creation", "file_number": 249997, "file_size": 37186806, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 3853849646, "table_properties": {"data_size": 30954912, "index_size": 1779072, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26012168, "raw_average_key_size": 28, "raw_value_size": 108205648, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929006, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732317173, "oldest_key_time": 0, "file_creation_time": 1732317173, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249997, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:12:56.592292 140025020393024 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249998.sst) /dev/shm/ToplingTemp/Topling-444076-j6uqa7 2024/11/23-07:12:57.468216 140025020393024 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249998.sst): first pass time = 0.88's, 153.237'MB/sec 2024/11/23-07:12:58.781335 140025020393024 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249998.sst): old prealloc_size = 147639500, real_size = 32506480 2024/11/23-07:12:58.781893 140025020393024 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249998.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932590 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0305 GB avg-val = 115.92 avg-zval = 32.70 2024/11/23-07:12:58.784320 140025020393024 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249998.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:12:59.736883 140025020393024 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249998.sst): second pass time = 1.31's, 82.369'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 38.325'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3320.727'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.282'MB/sec write SST data time = 0.13's, 283.747'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33290.127'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.369'MB/sec zip pipeline throughput = 82.369'MB/sec entries = 932590 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.92 avg-zval = 32.70 usrkeys = 932590 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.92 avg-zval = 32.70 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932590 bytes = 3264080 } seq expand size = 6528130 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5127 value = 3.5446 dict = 1.00 all = 3.6651 } Zip/UnZip{ index = 0.0951 value = 0.2821 dict = 1.00 all = 0.2728 } ---------------------------- total value len = 1.594666 GB avg = 0.123 KB (by entry num) total key len = 0.363969 GB avg = 0.028 KB total ukey len = 0.259978 GB avg = 0.020 KB total ukey num = 0.012998903 Billion total entry num = 0.012998903 Billion write speed all = 0.024350037 MB/sec (with seq num) write speed all = 0.023057203 MB/sec (without seq num) 2024/11/23-07:12:59.777588 140025020393024 [compaction_job.cc:1948] [default] [JOB 21644] Generated table #249998: 932590 keys, 36620916 bytes, temperature: kWarm 2024/11/23-07:12:59.777657 140025020393024 EVENT_LOG_v1 {"time_micros": 1732317179777626, "cf_name": "default", "job": 21644, "event": "table_file_creation", "file_number": 249998, "file_size": 36620916, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3621075956, "largest_seqno": 3853446381, "table_properties": {"data_size": 30498912, "index_size": 1774208, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26112520, "raw_average_key_size": 28, "raw_value_size": 108105240, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932590, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732317176, "oldest_key_time": 0, "file_creation_time": 1732317176, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249998, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:12:59.813165 140025020393024 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249999.sst) /dev/shm/ToplingTemp/Topling-444076-JSK2D7 2024/11/23-07:12:59.914400 140025020393024 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249999.sst): first pass time = 0.10's, 236.752'MB/sec 2024/11/23-07:13:00.138643 140025020393024 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249999.sst): old prealloc_size = 147639500, real_size = 6443072 2024/11/23-07:13:00.138802 140025020393024 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249999.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 168599 prefix = 4 raw-key = 0.0027 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0192 GB zip-val = 0.0061 GB avg-val = 114.11 avg-zval = 36.04 2024/11/23-07:13:00.138978 140025020393024 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249999.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0004 GB, waited 0.000 sec, Key+Value = 0.024 GB 2024/11/23-07:13:00.295940 140025020393024 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00/249999.sst): second pass time = 0.22's, 86.015'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 45.883'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2349.914'MB/sec (index lex order gen) rebuild zvType time = 0.13's, 0.315'MB/sec write SST data time = 0.02's, 309.602'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.202'MB, throughput = 7515.469'MB/sec zip my value time = 0.22's, unzip length = 0.019'GB zip my value throughput = 86.015'MB/sec zip pipeline throughput = 86.015'MB/sec entries = 168599 avg-key = 28.00 avg-zkey = 1.92 avg-val = 114.11 avg-zval = 36.04 usrkeys = 168599 avg-key = 20.00 avg-zkey = 1.92 avg-val = 121.11 avg-zval = 36.04 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 168599 bytes = 611184 } seq expand size = 1180193 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0034 GB value = 0.0192 GB dict = 0.20 MB all = 0.0240 GB } __ZipSize{ index = 0.0003 GB value = 0.0061 GB dict = 0.20 MB all = 0.0072 GB } UnZip/Zip{ index = 10.4063 value = 3.1659 dict = 1.00 all = 3.3200 } Zip/UnZip{ index = 0.0961 value = 0.3159 dict = 1.00 all = 0.3012 } ---------------------------- total value len = 1.615084 GB avg = 0.123 KB (by entry num) total key len = 0.368690 GB avg = 0.028 KB total ukey len = 0.263350 GB avg = 0.020 KB total ukey num = 0.013167502 Billion total entry num = 0.013167502 Billion write speed all = 0.024662399 MB/sec (with seq num) write speed all = 0.023352806 MB/sec (without seq num) 2024/11/23-07:13:00.307519 140025020393024 [compaction_job.cc:1948] [default] [JOB 21644] Generated table #249999: 168599 keys, 7216513 bytes, temperature: kWarm 2024/11/23-07:13:00.307579 140025020393024 EVENT_LOG_v1 {"time_micros": 1732317180307547, "cf_name": "default", "job": 21644, "event": "table_file_creation", "file_number": 249999, "file_size": 7216513, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3852354574, "table_properties": {"data_size": 6076720, "index_size": 324032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4720772, "raw_average_key_size": 28, "raw_value_size": 19238118, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 168599, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732317179, "oldest_key_time": 0, "file_creation_time": 1732317179, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249999, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:13:00.318207 140025020393024 [compaction_job.cc:703] job-21644: subcompact[0], size: 0.527069 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21644/att-00 [249985,249986,249987,249988,249989,249990,249991,249992,249993,249994,249995,249996,249997,249998,249999] 2024/11/23-07:13:00.415181 140025020393024 (Original Log Time 2024/11/23-07:12:07.007159) EVENT_LOG_v1 {"time_micros": 1732317127007115, "job": 21644, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [249973], "files_L3": [249891, 249892, 249893, 249894, 249895, 249896, 249897, 249898, 249899, 249900, 249901, 249933, 249934, 249935], "score": 0.983544, "input_data_size": 546213962, "oldest_snapshot_seqno": 3853871931} 2024/11/23-07:13:00.415185 140025020393024 (Original Log Time 2024/11/23-07:13:00.414355) [compaction_job.cc:2022] [default] [JOB 21644] Compacted 1@2 + 14@3 files to L3 => 527069299 bytes 2024/11/23-07:13:00.415186 140025020393024 (Original Log Time 2024/11/23-07:13:00.415113) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 10.2 rd, 9.9 wr, level 3, files in(1, 14) out(15 +0 blob) MB in(27.2, 493.7 +0.0 blob) out(502.7 +0.0 blob), read-write-amplify(37.6) write-amplify(18.5) OK, records in: 13235571, records dropped: 68069 output_compression: Snappy 2024/11/23-07:13:00.415192 140025020393024 (Original Log Time 2024/11/23-07:13:00.415159) EVENT_LOG_v1 {"time_micros": 1732317180415135, "cf": "default", "job": 21644, "event": "compaction_finished", "compaction_time_micros": 53311055, "compaction_time_cpu_micros": 22022421, "output_level": 3, "num_output_files": 15, "total_output_size": 527069299, "num_input_records": 13235571, "num_output_records": 13167502, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 338432155, "file_range_sync_nanos": 0, "file_fsync_nanos": 13908, "file_prepare_write_nanos": 35145, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/23-07:13:00.549593 140025020393024 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732317126,"executesMs":53537,"compactionJobId":21644,"attempt":0,"compactionInputRawBytes":1912674022,"compactionInputZipBytes":546213962,"compactionOutputRawBytes":0,"compactionOutputZipBytes":527069299,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-9898406abd308a40faaef5dd214745bf-0e11cdb45fdbdaf9-00"}}, response = 2024/11/23-07:13:00.549681 140025020393024 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-21644/att-00: olev 3, work 53.538 s, result 6.380 ms, install 0.863 ms, input{raw 1.781 GiB zip 520.910 MiB}