2024/11/23-07:28:11.008552 140025230210624 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00: fork to child time = 0.023940 sec 2024/11/23-07:28:11.009684 140025230210624 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00 2024/11/23-07:28:11.018999 140025230210624 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00 2024/11/23-07:28:11.020450 140025230210624 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-07:28:11.020837 140025230210624 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-07:28:11.066630 140025230210624 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-21854/att-00/MANIFEST-206347 2024/11/23-07:28:11.110491 140025230210624 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-21854/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 250492, last_sequence is 3860174040, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 250294 2024/11/23-07:28:11.110501 140025230210624 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-07:28:11.110629 140025230210624 [version_set.cc:5721] Creating manifest 250492 2024/11/23-07:28:11.141472 140025230210624 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-07:28:11.160495 140025230210624 [compaction_job.cc:2437] [default] [JOB 21854] Compacting 1@2 + 16@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-07:28:11.160518 140025230210624 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 16394 Base level 2, inputs: [250431(27MB)], [250490(8213KB) 250395(34MB) 250396(32MB) 250397(35MB) 250398(34MB) 250399(34MB) 250400(34MB) 250401(35MB) 250402(35MB) 250403(36MB) 250404(33MB) 250405(33MB) 250406(13MB) 250377(33MB) 250378(35MB) 250452(35MB)] 2024/11/23-07:28:11.170153 140025230210624 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-07:28:11.170187 140025230210624 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-07:28:11.170207 140025230210624 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-07:28:11.170213 140025230210624 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-07:28:11.170223 140025230210624 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-07:28:11.473839 140025230210624 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250493.sst) /dev/shm/ToplingTemp/Topling-447387-Htn4iu 2024/11/23-07:28:12.535904 140025230210624 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250493.sst): first pass time = 1.06's, 126.379'MB/sec 2024/11/23-07:28:13.992536 140025230210624 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250493.sst): old prealloc_size = 147639500, real_size = 31275248 2024/11/23-07:28:13.993263 140025230210624 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250493.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931446 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0293 GB avg-val = 116.10 avg-zval = 31.42 2024/11/23-07:28:13.994165 140025230210624 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250493.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:28:14.951299 140025230210624 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250493.sst): second pass time = 1.46's, 74.318'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 37.030'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2563.340'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.291'MB/sec write SST data time = 0.16's, 226.668'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35544.361'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 74.318'MB/sec zip pipeline throughput = 74.318'MB/sec entries = 931446 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.10 avg-zval = 31.42 usrkeys = 931446 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.10 avg-zval = 31.42 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931446 bytes = 3376512 } seq expand size = 6520122 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.5131 value = 3.6944 dict = 1.00 all = 3.7805 } Zip/UnZip{ index = 0.0951 value = 0.2707 dict = 1.00 all = 0.2645 } ---------------------------- total value len = 0.114657 GB avg = 0.123 KB (by entry num) total key len = 0.026080 GB avg = 0.028 KB total ukey len = 0.018629 GB avg = 0.020 KB total ukey num = 0.000931446 Billion total entry num = 0.000931446 Billion write speed all = 0.001729989 MB/sec (with seq num) write speed all = 0.001638393 MB/sec (without seq num) 2024/11/23-07:28:14.985850 140025230210624 [compaction_job.cc:1948] [default] [JOB 21854] Generated table #250493: 931446 keys, 35502880 bytes, temperature: kWarm 2024/11/23-07:28:14.985936 140025230210624 EVENT_LOG_v1 {"time_micros": 1732318094985884, "cf_name": "default", "job": 21854, "event": "table_file_creation", "file_number": 250493, "file_size": 35502880, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3859851660, "table_properties": {"data_size": 29270240, "index_size": 1771968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26080488, "raw_average_key_size": 28, "raw_value_size": 108137240, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931446, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732318091, "oldest_key_time": 0, "file_creation_time": 1732318091, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250493, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:28:15.021320 140025230210624 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250494.sst) /dev/shm/ToplingTemp/Topling-447387-TckA2v 2024/11/23-07:28:16.027805 140025230210624 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250494.sst): first pass time = 1.01's, 133.358'MB/sec 2024/11/23-07:28:17.643917 140025230210624 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250494.sst): old prealloc_size = 147639500, real_size = 31125232 2024/11/23-07:28:17.644491 140025230210624 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250494.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923176 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1084 GB zip-val = 0.0291 GB avg-val = 117.39 avg-zval = 31.53 2024/11/23-07:28:17.645281 140025230210624 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250494.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:28:18.491397 140025230210624 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250494.sst): second pass time = 1.62's, 67.091'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 41.687'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3239.562'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 277.603'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34100.271'MB/sec zip my value time = 1.62's, unzip length = 0.108'GB zip my value throughput = 67.091'MB/sec zip pipeline throughput = 92.983'MB/sec entries = 923176 avg-key = 28.00 avg-zkey = 1.94 avg-val = 117.39 avg-zval = 31.53 usrkeys = 923176 avg-key = 20.00 avg-zkey = 1.94 avg-val = 124.39 avg-zval = 31.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923176 bytes = 3346528 } seq expand size = 6462232 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0353 GB } UnZip/Zip{ index = 10.3107 value = 3.7236 dict = 1.00 all = 3.7992 } Zip/UnZip{ index = 0.0970 value = 0.2686 dict = 1.00 all = 0.2632 } ---------------------------- total value len = 0.229489 GB avg = 0.124 KB (by entry num) total key len = 0.051929 GB avg = 0.028 KB total ukey len = 0.037092 GB avg = 0.020 KB total ukey num = 0.001854622 Billion total entry num = 0.001854622 Billion write speed all = 0.003459118 MB/sec (with seq num) write speed all = 0.003276746 MB/sec (without seq num) 2024/11/23-07:28:18.527679 140025230210624 [compaction_job.cc:1948] [default] [JOB 21854] Generated table #250494: 923176 keys, 35327828 bytes, temperature: kWarm 2024/11/23-07:28:18.527742 140025230210624 EVENT_LOG_v1 {"time_micros": 1732318098527710, "cf_name": "default", "job": 21854, "event": "table_file_creation", "file_number": 250494, "file_size": 35327828, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3377377158, "largest_seqno": 3859851051, "table_properties": {"data_size": 29103520, "index_size": 1790720, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25848928, "raw_average_key_size": 28, "raw_value_size": 108368920, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923176, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732318095, "oldest_key_time": 0, "file_creation_time": 1732318095, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250494, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:28:18.571983 140025230210624 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250495.sst) /dev/shm/ToplingTemp/Topling-447387-OZuosu 2024/11/23-07:28:19.585224 140025230210624 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250495.sst): first pass time = 1.01's, 132.471'MB/sec 2024/11/23-07:28:21.249554 140025230210624 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250495.sst): old prealloc_size = 147639500, real_size = 33818832 2024/11/23-07:28:21.250106 140025230210624 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250495.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931388 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0318 GB avg-val = 116.11 avg-zval = 34.16 2024/11/23-07:28:21.250928 140025230210624 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250495.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:28:22.385234 140025230210624 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250495.sst): second pass time = 1.66's, 65.001'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.14's, 33.501'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3350.442'MB/sec (index lex order gen) rebuild zvType time = 0.88's, 0.265'MB/sec write SST data time = 0.26's, 147.793'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35140.035'MB/sec zip my value time = 1.66's, unzip length = 0.108'GB zip my value throughput = 65.001'MB/sec zip pipeline throughput = 120.714'MB/sec entries = 931388 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.11 avg-zval = 34.16 usrkeys = 931388 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.11 avg-zval = 34.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931388 bytes = 3376304 } seq expand size = 6519716 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.5121 value = 3.3991 dict = 1.00 all = 3.5278 } Zip/UnZip{ index = 0.0951 value = 0.2942 dict = 1.00 all = 0.2835 } ---------------------------- total value len = 0.344147 GB avg = 0.124 KB (by entry num) total key len = 0.078008 GB avg = 0.028 KB total ukey len = 0.055720 GB avg = 0.020 KB total ukey num = 0.002786010 Billion total entry num = 0.002786010 Billion write speed all = 0.005188779 MB/sec (with seq num) write speed all = 0.004914833 MB/sec (without seq num) 2024/11/23-07:28:22.423411 140025230210624 [compaction_job.cc:1948] [default] [JOB 21854] Generated table #250495: 931388 keys, 38045860 bytes, temperature: kWarm 2024/11/23-07:28:22.423484 140025230210624 EVENT_LOG_v1 {"time_micros": 1732318102423443, "cf_name": "default", "job": 21854, "event": "table_file_creation", "file_number": 250495, "file_size": 38045860, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3563652023, "largest_seqno": 3859851825, "table_properties": {"data_size": 31813760, "index_size": 1772032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26078864, "raw_average_key_size": 28, "raw_value_size": 108138888, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931388, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732318098, "oldest_key_time": 0, "file_creation_time": 1732318098, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250495, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:28:22.461033 140025230210624 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250496.sst) /dev/shm/ToplingTemp/Topling-447387-Cz379t 2024/11/23-07:28:23.300867 140025230210624 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250496.sst): first pass time = 0.84's, 159.822'MB/sec 2024/11/23-07:28:24.624021 140025230210624 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250496.sst): old prealloc_size = 147639500, real_size = 32325488 2024/11/23-07:28:24.624579 140025230210624 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250496.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933621 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0303 GB avg-val = 115.76 avg-zval = 32.46 2024/11/23-07:28:24.625328 140025230210624 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250496.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:28:25.566140 140025230210624 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250496.sst): second pass time = 1.32's, 81.721'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 38.436'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3327.700'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.288'MB/sec write SST data time = 0.13's, 276.609'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32734.714'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.721'MB/sec zip pipeline throughput = 81.721'MB/sec entries = 933621 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.76 avg-zval = 32.46 usrkeys = 933621 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.76 avg-zval = 32.46 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933621 bytes = 3034288 } seq expand size = 6535347 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.4456 value = 3.5664 dict = 1.00 all = 3.7066 } Zip/UnZip{ index = 0.0957 value = 0.2804 dict = 1.00 all = 0.2698 } ---------------------------- total value len = 0.458759 GB avg = 0.123 KB (by entry num) total key len = 0.104150 GB avg = 0.028 KB total ukey len = 0.074393 GB avg = 0.020 KB total ukey num = 0.003719631 Billion total entry num = 0.003719631 Billion write speed all = 0.006918528 MB/sec (with seq num) write speed all = 0.006552794 MB/sec (without seq num) 2024/11/23-07:28:25.601842 140025230210624 [compaction_job.cc:1948] [default] [JOB 21854] Generated table #250496: 933621 keys, 36210456 bytes, temperature: kWarm 2024/11/23-07:28:25.601907 140025230210624 EVENT_LOG_v1 {"time_micros": 1732318105601873, "cf_name": "default", "job": 21854, "event": "table_file_creation", "file_number": 250496, "file_size": 36210456, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3795071236, "largest_seqno": 3859851215, "table_properties": {"data_size": 30304288, "index_size": 1787584, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26141388, "raw_average_key_size": 28, "raw_value_size": 108076402, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933621, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732318102, "oldest_key_time": 0, "file_creation_time": 1732318102, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250496, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:28:25.636258 140025230210624 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250497.sst) /dev/shm/ToplingTemp/Topling-447387-IJ0Npx 2024/11/23-07:28:26.476540 140025230210624 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250497.sst): first pass time = 0.84's, 159.737'MB/sec 2024/11/23-07:28:27.859400 140025230210624 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250497.sst): old prealloc_size = 147639500, real_size = 35114064 2024/11/23-07:28:27.860037 140025230210624 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250497.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1082093 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1039 GB zip-val = 0.0328 GB avg-val = 96.04 avg-zval = 30.27 2024/11/23-07:28:27.860222 140025230210624 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250497.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:28:28.488123 140025230210624 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250497.sst): second pass time = 1.38's, 75.186'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.63's, 57.480'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3398.291'MB/sec (index lex order gen) rebuild zvType time = 0.49's, 0.555'MB/sec write SST data time = 0.14's, 257.670'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.032'MB, throughput = 30799.934'MB/sec zip my value time = 1.38's, unzip length = 0.104'GB zip my value throughput = 75.186'MB/sec zip pipeline throughput = 75.186'MB/sec entries = 1082093 avg-key = 28.00 avg-zkey = 1.93 avg-val = 96.04 avg-zval = 30.27 usrkeys = 1082093 avg-key = 20.00 avg-zkey = 1.93 avg-val = 96.22 avg-zval = 30.27 TagRS{ kinds = RS_Key0_TagN bytes = 169160 } TagArray{ size = 28858 bytes = 82976 } seq expand size = 202006 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0216 GB value = 0.1039 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0328 GB dict = 1.03 MB all = 0.0361 GB } UnZip/Zip{ index = 10.3792 value = 3.1723 dict = 1.00 all = 3.7140 } Zip/UnZip{ index = 0.0963 value = 0.3152 dict = 1.00 all = 0.2693 } ---------------------------- total value len = 0.562880 GB avg = 0.117 KB (by entry num) total key len = 0.134448 GB avg = 0.028 KB total ukey len = 0.096034 GB avg = 0.020 KB total ukey num = 0.004801724 Billion total entry num = 0.004801724 Billion write speed all = 0.008570331 MB/sec (with seq num) write speed all = 0.008098216 MB/sec (without seq num) 2024/11/23-07:28:28.524840 140025230210624 [compaction_job.cc:1948] [default] [JOB 21854] Generated table #250497: 1082093 keys, 36138612 bytes, temperature: kWarm 2024/11/23-07:28:28.524903 140025230210624 EVENT_LOG_v1 {"time_micros": 1732318108524871, "cf_name": "default", "job": 21854, "event": "table_file_creation", "file_number": 250497, "file_size": 36138612, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3859850298, "table_properties": {"data_size": 32758208, "index_size": 2085120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30298604, "raw_average_key_size": 28, "raw_value_size": 103919183, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1082093, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732318105, "oldest_key_time": 0, "file_creation_time": 1732318105, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250497, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:28:28.563404 140025230210624 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250498.sst) /dev/shm/ToplingTemp/Topling-447387-aQs8mv 2024/11/23-07:28:29.403498 140025230210624 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250498.sst): first pass time = 0.84's, 159.772'MB/sec 2024/11/23-07:28:30.768941 140025230210624 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250498.sst): old prealloc_size = 147639500, real_size = 34828416 2024/11/23-07:28:30.769493 140025230210624 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250498.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922119 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0328 GB avg-val = 117.55 avg-zval = 35.62 2024/11/23-07:28:30.770087 140025230210624 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250498.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:28:31.618308 140025230210624 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250498.sst): second pass time = 1.36's, 79.429'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.782'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3332.657'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 291.453'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33544.779'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.429'MB/sec zip pipeline throughput = 79.429'MB/sec entries = 922119 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.55 avg-zval = 35.62 usrkeys = 922119 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.01 avg-zval = 35.62 TagRS{ kinds = RS_Key0_TagN bytes = 144200 } TagArray{ size = 586951 bytes = 2201088 } seq expand size = 4108657 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0328 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.5126 value = 3.3005 dict = 1.00 all = 3.5287 } Zip/UnZip{ index = 0.0951 value = 0.3030 dict = 1.00 all = 0.2834 } ---------------------------- total value len = 0.675387 GB avg = 0.118 KB (by entry num) total key len = 0.160268 GB avg = 0.028 KB total ukey len = 0.114477 GB avg = 0.020 KB total ukey num = 0.005723843 Billion total entry num = 0.005723843 Billion write speed all = 0.010270001 MB/sec (with seq num) write speed all = 0.009707244 MB/sec (without seq num) 2024/11/23-07:28:31.655988 140025230210624 [compaction_job.cc:1948] [default] [JOB 21854] Generated table #250498: 922119 keys, 38035948 bytes, temperature: kWarm 2024/11/23-07:28:31.656054 140025230210624 EVENT_LOG_v1 {"time_micros": 1732318111656022, "cf_name": "default", "job": 21854, "event": "table_file_creation", "file_number": 250498, "file_size": 38035948, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3849588582, "table_properties": {"data_size": 32843376, "index_size": 1754304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25819332, "raw_average_key_size": 28, "raw_value_size": 108398518, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922119, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732318108, "oldest_key_time": 0, "file_creation_time": 1732318108, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250498, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:28:31.691352 140025230210624 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250499.sst) /dev/shm/ToplingTemp/Topling-447387-f9m6ax 2024/11/23-07:28:32.495710 140025230210624 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250499.sst): first pass time = 0.80's, 166.872'MB/sec 2024/11/23-07:28:33.816370 140025230210624 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250499.sst): old prealloc_size = 147639500, real_size = 32478832 2024/11/23-07:28:33.816947 140025230210624 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250499.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933807 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0305 GB avg-val = 115.73 avg-zval = 32.63 2024/11/23-07:28:33.818769 140025230210624 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250499.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:28:34.778669 140025230210624 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250499.sst): second pass time = 1.32's, 81.909'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 38.153'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3170.551'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.281'MB/sec write SST data time = 0.13's, 277.528'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36585.089'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.909'MB/sec zip pipeline throughput = 81.909'MB/sec entries = 933807 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.73 avg-zval = 32.63 usrkeys = 933807 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.73 avg-zval = 32.63 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933807 bytes = 3385072 } seq expand size = 6536649 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5117 value = 3.5470 dict = 1.00 all = 3.6557 } Zip/UnZip{ index = 0.0951 value = 0.2819 dict = 1.00 all = 0.2735 } ---------------------------- total value len = 0.789995 GB avg = 0.119 KB (by entry num) total key len = 0.186414 GB avg = 0.028 KB total ukey len = 0.133153 GB avg = 0.020 KB total ukey num = 0.006657650 Billion total entry num = 0.006657650 Billion write speed all = 0.011999374 MB/sec (with seq num) write speed all = 0.011344832 MB/sec (without seq num) 2024/11/23-07:28:34.824130 140025230210624 [compaction_job.cc:1948] [default] [JOB 21854] Generated table #250499: 933807 keys, 36714494 bytes, temperature: kWarm 2024/11/23-07:28:34.824195 140025230210624 EVENT_LOG_v1 {"time_micros": 1732318114824162, "cf_name": "default", "job": 21854, "event": "table_file_creation", "file_number": 250499, "file_size": 36714494, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3487497056, "largest_seqno": 3859852077, "table_properties": {"data_size": 30468512, "index_size": 1776704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26146596, "raw_average_key_size": 28, "raw_value_size": 108071158, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933807, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732318111, "oldest_key_time": 0, "file_creation_time": 1732318111, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250499, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:28:34.857526 140025230210624 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250500.sst) /dev/shm/ToplingTemp/Topling-447387-sdRSQw 2024/11/23-07:28:35.655954 140025230210624 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250500.sst): first pass time = 0.80's, 168.111'MB/sec 2024/11/23-07:28:36.980524 140025230210624 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250500.sst): old prealloc_size = 147639500, real_size = 33349040 2024/11/23-07:28:36.981074 140025230210624 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250500.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932182 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0313 GB avg-val = 115.98 avg-zval = 33.62 2024/11/23-07:28:36.983190 140025230210624 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250500.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:28:37.896811 140025230210624 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250500.sst): second pass time = 1.32's, 81.694'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 41.140'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3357.428'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.296'MB/sec write SST data time = 0.13's, 290.678'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32115.243'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.694'MB/sec zip pipeline throughput = 81.694'MB/sec entries = 932182 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.98 avg-zval = 33.62 usrkeys = 932182 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.98 avg-zval = 33.62 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932182 bytes = 3495696 } seq expand size = 6525274 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5119 value = 3.4496 dict = 1.00 all = 3.5606 } Zip/UnZip{ index = 0.0951 value = 0.2899 dict = 1.00 all = 0.2809 } ---------------------------- total value len = 0.904637 GB avg = 0.119 KB (by entry num) total key len = 0.212515 GB avg = 0.028 KB total ukey len = 0.151797 GB avg = 0.020 KB total ukey num = 0.007589832 Billion total entry num = 0.007589832 Billion write speed all = 0.013728479 MB/sec (with seq num) write speed all = 0.012982319 MB/sec (without seq num) 2024/11/23-07:28:37.932736 140025230210624 [compaction_job.cc:1948] [default] [JOB 21854] Generated table #250500: 932182 keys, 37695468 bytes, temperature: kWarm 2024/11/23-07:28:37.932805 140025230210624 EVENT_LOG_v1 {"time_micros": 1732318117932768, "cf_name": "default", "job": 21854, "event": "table_file_creation", "file_number": 250500, "file_size": 37695468, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3859851273, "table_properties": {"data_size": 31342240, "index_size": 1773568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26101096, "raw_average_key_size": 28, "raw_value_size": 108116636, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932182, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732318114, "oldest_key_time": 0, "file_creation_time": 1732318114, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250500, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:28:37.970441 140025230210624 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250501.sst) /dev/shm/ToplingTemp/Topling-447387-XsIHPv 2024/11/23-07:28:38.791404 140025230210624 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250501.sst): first pass time = 0.82's, 163.496'MB/sec 2024/11/23-07:28:40.095724 140025230210624 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250501.sst): old prealloc_size = 147639500, real_size = 32910064 2024/11/23-07:28:40.096277 140025230210624 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250501.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930103 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.30 avg-zval = 33.23 2024/11/23-07:28:40.098108 140025230210624 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250501.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:28:41.034600 140025230210624 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250501.sst): second pass time = 1.30's, 83.027'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.428'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3321.301'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.287'MB/sec write SST data time = 0.13's, 287.482'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 37112.258'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.027'MB/sec zip pipeline throughput = 83.027'MB/sec entries = 930103 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.30 avg-zval = 33.23 usrkeys = 930103 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.30 avg-zval = 33.23 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930103 bytes = 3255376 } seq expand size = 6510721 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5124 value = 3.4999 dict = 1.00 all = 3.6258 } Zip/UnZip{ index = 0.0951 value = 0.2857 dict = 1.00 all = 0.2758 } ---------------------------- total value len = 1.019323 GB avg = 0.120 KB (by entry num) total key len = 0.238558 GB avg = 0.028 KB total ukey len = 0.170399 GB avg = 0.020 KB total ukey num = 0.008519935 Billion total entry num = 0.008519935 Billion write speed all = 0.015457271 MB/sec (with seq num) write speed all = 0.014619704 MB/sec (without seq num) 2024/11/23-07:28:41.069995 140025230210624 [compaction_job.cc:1948] [default] [JOB 21854] Generated table #250501: 930103 keys, 37017844 bytes, temperature: kWarm 2024/11/23-07:28:41.070058 140025230210624 EVENT_LOG_v1 {"time_micros": 1732318121070027, "cf_name": "default", "job": 21854, "event": "table_file_creation", "file_number": 250501, "file_size": 37017844, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3632256661, "largest_seqno": 3859850600, "table_properties": {"data_size": 30907808, "index_size": 1769536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26042884, "raw_average_key_size": 28, "raw_value_size": 108174982, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930103, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732318117, "oldest_key_time": 0, "file_creation_time": 1732318117, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250501, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:28:41.110984 140025230210624 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250502.sst) /dev/shm/ToplingTemp/Topling-447387-HDgNhw 2024/11/23-07:28:41.920111 140025230210624 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250502.sst): first pass time = 0.81's, 165.887'MB/sec 2024/11/23-07:28:43.228602 140025230210624 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250502.sst): old prealloc_size = 147639500, real_size = 32679440 2024/11/23-07:28:43.229132 140025230210624 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250502.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924774 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0307 GB avg-val = 117.14 avg-zval = 33.18 2024/11/23-07:28:43.230923 140025230210624 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250502.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:28:44.144214 140025230210624 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250502.sst): second pass time = 1.31's, 82.826'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.288'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3475.848'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.294'MB/sec write SST data time = 0.13's, 285.565'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32121.241'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.826'MB/sec zip pipeline throughput = 82.826'MB/sec entries = 924774 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.14 avg-zval = 33.18 usrkeys = 924774 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.14 avg-zval = 33.18 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924774 bytes = 3352320 } seq expand size = 6473418 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5119 value = 3.5298 dict = 1.00 all = 3.6386 } Zip/UnZip{ index = 0.0951 value = 0.2833 dict = 1.00 all = 0.2748 } ---------------------------- total value len = 1.134120 GB avg = 0.120 KB (by entry num) total key len = 0.264452 GB avg = 0.028 KB total ukey len = 0.188894 GB avg = 0.020 KB total ukey num = 0.009444709 Billion total entry num = 0.009444709 Billion write speed all = 0.017185476 MB/sec (with seq num) write speed all = 0.016257033 MB/sec (without seq num) 2024/11/23-07:28:44.179500 140025230210624 [compaction_job.cc:1948] [default] [JOB 21854] Generated table #250502: 924774 keys, 36887360 bytes, temperature: kWarm 2024/11/23-07:28:44.179560 140025230210624 EVENT_LOG_v1 {"time_micros": 1732318124179528, "cf_name": "default", "job": 21854, "event": "table_file_creation", "file_number": 250502, "file_size": 36887360, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369194706, "largest_seqno": 3859851708, "table_properties": {"data_size": 30688576, "index_size": 1759488, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25893672, "raw_average_key_size": 28, "raw_value_size": 108324116, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924774, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732318121, "oldest_key_time": 0, "file_creation_time": 1732318121, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250502, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:28:44.215719 140025230210624 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250503.sst) /dev/shm/ToplingTemp/Topling-447387-ZNJgfx 2024/11/23-07:28:45.099248 140025230210624 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250503.sst): first pass time = 0.88's, 151.917'MB/sec 2024/11/23-07:28:46.393525 140025230210624 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250503.sst): old prealloc_size = 147639500, real_size = 33086736 2024/11/23-07:28:46.394055 140025230210624 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250503.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927739 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.67 avg-zval = 33.51 2024/11/23-07:28:46.394843 140025230210624 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250503.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:28:47.353028 140025230210624 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250503.sst): second pass time = 1.29's, 83.691'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 38.879'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3486.381'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.279'MB/sec write SST data time = 0.13's, 292.228'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33507.881'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.691'MB/sec zip pipeline throughput = 83.691'MB/sec entries = 927739 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.67 avg-zval = 33.51 usrkeys = 927739 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.67 avg-zval = 33.51 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927739 bytes = 3363072 } seq expand size = 6494173 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5131 value = 3.4816 dict = 1.00 all = 3.5980 } Zip/UnZip{ index = 0.0951 value = 0.2872 dict = 1.00 all = 0.2779 } ---------------------------- total value len = 1.248855 GB avg = 0.120 KB (by entry num) total key len = 0.290429 GB avg = 0.028 KB total ukey len = 0.207449 GB avg = 0.020 KB total ukey num = 0.010372448 Billion total entry num = 0.010372448 Billion write speed all = 0.018913781 MB/sec (with seq num) write speed all = 0.017894178 MB/sec (without seq num) 2024/11/23-07:28:47.389592 140025230210624 [compaction_job.cc:1948] [default] [JOB 21854] Generated table #250503: 927739 keys, 37303588 bytes, temperature: kWarm 2024/11/23-07:28:47.389653 140025230210624 EVENT_LOG_v1 {"time_micros": 1732318127389622, "cf_name": "default", "job": 21854, "event": "table_file_creation", "file_number": 250503, "file_size": 37303588, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 3859850099, "table_properties": {"data_size": 31089664, "index_size": 1764928, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25976692, "raw_average_key_size": 28, "raw_value_size": 108241074, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927739, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732318124, "oldest_key_time": 0, "file_creation_time": 1732318124, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250503, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:28:47.421871 140025230210624 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250504.sst) /dev/shm/ToplingTemp/Topling-447387-swTgyv 2024/11/23-07:28:48.336078 140025230210624 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250504.sst): first pass time = 0.91's, 146.820'MB/sec 2024/11/23-07:28:49.675589 140025230210624 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250504.sst): old prealloc_size = 147639500, real_size = 33308272 2024/11/23-07:28:49.676148 140025230210624 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250504.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928753 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0313 GB avg-val = 116.51 avg-zval = 33.71 2024/11/23-07:28:49.678095 140025230210624 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250504.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:28:50.638173 140025230210624 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250504.sst): second pass time = 1.34's, 80.826'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 38.867'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3275.888'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.279'MB/sec write SST data time = 0.13's, 289.070'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32370.812'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.826'MB/sec zip pipeline throughput = 80.826'MB/sec entries = 928753 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.51 avg-zval = 33.71 usrkeys = 928753 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.51 avg-zval = 33.71 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928753 bytes = 3250656 } seq expand size = 6501271 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5120 value = 3.4563 dict = 1.00 all = 3.5876 } Zip/UnZip{ index = 0.0951 value = 0.2893 dict = 1.00 all = 0.2787 } ---------------------------- total value len = 1.363569 GB avg = 0.121 KB (by entry num) total key len = 0.316434 GB avg = 0.028 KB total ukey len = 0.226024 GB avg = 0.020 KB total ukey num = 0.011301201 Billion total entry num = 0.011301201 Billion write speed all = 0.020642017 MB/sec (with seq num) write speed all = 0.019531164 MB/sec (without seq num) 2024/11/23-07:28:50.676015 140025230210624 [compaction_job.cc:1948] [default] [JOB 21854] Generated table #250504: 928753 keys, 37411900 bytes, temperature: kWarm 2024/11/23-07:28:50.676083 140025230210624 EVENT_LOG_v1 {"time_micros": 1732318130676044, "cf_name": "default", "job": 21854, "event": "table_file_creation", "file_number": 250504, "file_size": 37411900, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3610601794, "largest_seqno": 3859851795, "table_properties": {"data_size": 31308832, "index_size": 1767040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26005084, "raw_average_key_size": 28, "raw_value_size": 108212698, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928753, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732318127, "oldest_key_time": 0, "file_creation_time": 1732318127, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250504, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:28:50.713758 140025230210624 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250505.sst) /dev/shm/ToplingTemp/Topling-447387-3w1wBu 2024/11/23-07:28:51.526964 140025230210624 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250505.sst): first pass time = 0.81's, 165.057'MB/sec 2024/11/23-07:28:52.822283 140025230210624 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250505.sst): old prealloc_size = 147639500, real_size = 32674816 2024/11/23-07:28:52.822871 140025230210624 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250505.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933061 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0307 GB avg-val = 115.85 avg-zval = 32.87 2024/11/23-07:28:52.824923 140025230210624 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250505.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:28:53.767932 140025230210624 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250505.sst): second pass time = 1.29's, 83.491'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.030'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3152.778'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.287'MB/sec write SST data time = 0.13's, 281.891'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31618.067'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.491'MB/sec zip pipeline throughput = 83.491'MB/sec entries = 933061 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.85 avg-zval = 32.87 usrkeys = 933061 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.85 avg-zval = 32.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933061 bytes = 3382368 } seq expand size = 6531427 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5135 value = 3.5248 dict = 1.00 all = 3.6366 } Zip/UnZip{ index = 0.0951 value = 0.2837 dict = 1.00 all = 0.2750 } ---------------------------- total value len = 1.478193 GB avg = 0.121 KB (by entry num) total key len = 0.342559 GB avg = 0.028 KB total ukey len = 0.244685 GB avg = 0.020 KB total ukey num = 0.012234262 Billion total entry num = 0.012234262 Billion write speed all = 0.022370527 MB/sec (with seq num) write speed all = 0.021168005 MB/sec (without seq num) 2024/11/23-07:28:53.803218 140025230210624 [compaction_job.cc:1948] [default] [JOB 21854] Generated table #250505: 933061 keys, 36907858 bytes, temperature: kWarm 2024/11/23-07:28:53.803280 140025230210624 EVENT_LOG_v1 {"time_micros": 1732318133803250, "cf_name": "default", "job": 21854, "event": "table_file_creation", "file_number": 250505, "file_size": 36907858, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 3859851977, "table_properties": {"data_size": 30666416, "index_size": 1774976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26125708, "raw_average_key_size": 28, "raw_value_size": 108092046, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933061, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732318130, "oldest_key_time": 0, "file_creation_time": 1732318130, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250505, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:28:53.835971 140025230210624 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250506.sst) /dev/shm/ToplingTemp/Topling-447387-c7Ywxu 2024/11/23-07:28:54.665280 140025230210624 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250506.sst): first pass time = 0.83's, 161.851'MB/sec 2024/11/23-07:28:55.954250 140025230210624 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250506.sst): old prealloc_size = 147639500, real_size = 31216144 2024/11/23-07:28:55.954773 140025230210624 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250506.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920072 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0292 GB avg-val = 117.88 avg-zval = 31.76 2024/11/23-07:28:55.955556 140025230210624 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250506.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:28:56.795303 140025230210624 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250506.sst): second pass time = 1.29's, 84.189'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 41.966'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3437.577'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.324'MB/sec write SST data time = 0.13's, 272.446'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 24973.400'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.189'MB/sec zip pipeline throughput = 84.189'MB/sec entries = 920072 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.88 avg-zval = 31.76 usrkeys = 920072 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.88 avg-zval = 31.76 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 920072 bytes = 3220272 } seq expand size = 6440504 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4459 value = 3.7111 dict = 1.00 all = 3.8028 } Zip/UnZip{ index = 0.0957 value = 0.2695 dict = 1.00 all = 0.2630 } ---------------------------- total value len = 1.593089 GB avg = 0.121 KB (by entry num) total key len = 0.368321 GB avg = 0.028 KB total ukey len = 0.263087 GB avg = 0.020 KB total ukey num = 0.013154334 Billion total entry num = 0.013154334 Billion write speed all = 0.024097817 MB/sec (with seq num) write speed all = 0.022804908 MB/sec (without seq num) 2024/11/23-07:28:56.830181 140025230210624 [compaction_job.cc:1948] [default] [JOB 21854] Generated table #250506: 920072 keys, 35294690 bytes, temperature: kWarm 2024/11/23-07:28:56.830243 140025230210624 EVENT_LOG_v1 {"time_micros": 1732318136830210, "cf_name": "default", "job": 21854, "event": "table_file_creation", "file_number": 250506, "file_size": 35294690, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3675534142, "largest_seqno": 3859850674, "table_properties": {"data_size": 29224320, "index_size": 1761600, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25762016, "raw_average_key_size": 28, "raw_value_size": 108455760, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920072, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732318133, "oldest_key_time": 0, "file_creation_time": 1732318133, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250506, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:28:56.867287 140025230210624 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250507.sst) /dev/shm/ToplingTemp/Topling-447387-fjzUFw 2024/11/23-07:28:57.246447 140025230210624 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250507.sst): first pass time = 0.38's, 250.340'MB/sec 2024/11/23-07:28:58.189278 140025230210624 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250507.sst): old prealloc_size = 147639500, real_size = 24979616 2024/11/23-07:28:58.189695 140025230210624 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250507.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 660807 prefix = 4 raw-key = 0.0106 GB zip-key = 0.0013 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0764 GB zip-val = 0.0235 GB avg-val = 115.63 avg-zval = 35.64 2024/11/23-07:28:58.191169 140025230210624 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250507.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0017 GB, waited 0.000 sec, Key+Value = 0.095 GB 2024/11/23-07:28:58.781383 140025230210624 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00/250507.sst): second pass time = 0.94's, 81.094'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.59's, 46.423'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3211.978'MB/sec (index lex order gen) rebuild zvType time = 0.50's, 0.332'MB/sec write SST data time = 0.09's, 291.154'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.770'MB, throughput = 25107.277'MB/sec zip my value time = 0.94's, unzip length = 0.076'GB zip my value throughput = 81.094'MB/sec zip pipeline throughput = 81.094'MB/sec entries = 660807 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.63 avg-zval = 35.64 usrkeys = 660807 avg-key = 20.00 avg-zkey = 1.92 avg-val = 120.69 avg-zval = 35.64 TagRS{ kinds = RS_Key0_TagN bytes = 103360 } TagArray{ size = 477810 bytes = 1791808 } seq expand size = 3344670 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0132 GB value = 0.0764 GB dict = 0.77 MB all = 0.0949 GB } __ZipSize{ index = 0.0013 GB value = 0.0235 GB dict = 0.77 MB all = 0.0275 GB } UnZip/Zip{ index = 10.4415 value = 3.2447 dict = 1.00 all = 3.4529 } Zip/UnZip{ index = 0.0958 value = 0.3082 dict = 1.00 all = 0.2896 } ---------------------------- total value len = 1.672841 GB avg = 0.121 KB (by entry num) total key len = 0.386824 GB avg = 0.028 KB total ukey len = 0.276303 GB avg = 0.020 KB total ukey num = 0.013815141 Billion total entry num = 0.013815141 Billion write speed all = 0.025304347 MB/sec (with seq num) write speed all = 0.023946522 MB/sec (without seq num) 2024/11/23-07:28:58.810655 140025230210624 [compaction_job.cc:1948] [default] [JOB 21854] Generated table #250507: 660807 keys, 27486579 bytes, temperature: kWarm 2024/11/23-07:28:58.810716 140025230210624 EVENT_LOG_v1 {"time_micros": 1732318138810684, "cf_name": "default", "job": 21854, "event": "table_file_creation", "file_number": 250507, "file_size": 27486579, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3859851882, "table_properties": {"data_size": 23548496, "index_size": 1265728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 18502596, "raw_average_key_size": 28, "raw_value_size": 76406906, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 660807, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732318136, "oldest_key_time": 0, "file_creation_time": 1732318136, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250507, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:28:58.830804 140025230210624 [compaction_job.cc:703] job-21854: subcompact[0], size: 0.541981 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21854/att-00 [250493,250494,250495,250496,250497,250498,250499,250500,250501,250502,250503,250504,250505,250506,250507] 2024/11/23-07:28:58.924751 140025230210624 (Original Log Time 2024/11/23-07:28:11.160585) EVENT_LOG_v1 {"time_micros": 1732318091160536, "job": 21854, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [250431], "files_L3": [250490, 250395, 250396, 250397, 250398, 250399, 250400, 250401, 250402, 250403, 250404, 250405, 250406, 250377, 250378, 250452], "score": 0.98839, "input_data_size": 561672296, "oldest_snapshot_seqno": 3860174012} 2024/11/23-07:28:58.924754 140025230210624 (Original Log Time 2024/11/23-07:28:58.923993) [compaction_job.cc:2022] [default] [JOB 21854] Compacted 1@2 + 16@3 files to L3 => 541981365 bytes 2024/11/23-07:28:58.924755 140025230210624 (Original Log Time 2024/11/23-07:28:58.924684) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 11.8 rd, 11.4 wr, level 3, files in(1, 16) out(15 +0 blob) MB in(27.1, 508.6 +0.0 blob) out(516.9 +0.0 blob), read-write-amplify(38.8) write-amplify(19.1) OK, records in: 13887372, records dropped: 72231 output_compression: Snappy 2024/11/23-07:28:58.924762 140025230210624 (Original Log Time 2024/11/23-07:28:58.924730) EVENT_LOG_v1 {"time_micros": 1732318138924709, "cf": "default", "job": 21854, "event": "compaction_finished", "compaction_time_micros": 47670229, "compaction_time_cpu_micros": 21798495, "output_level": 3, "num_output_files": 15, "total_output_size": 541981365, "num_input_records": 13887372, "num_output_records": 13815141, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 341926659, "file_range_sync_nanos": 0, "file_fsync_nanos": 14301, "file_prepare_write_nanos": 35715, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/23-07:28:59.071904 140025230210624 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732318090,"executesMs":47908,"compactionJobId":21854,"attempt":0,"compactionInputRawBytes":1984215277,"compactionInputZipBytes":561672296,"compactionOutputRawBytes":0,"compactionOutputZipBytes":541981365,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-7bea86898e85f423a1709d4f38ae3a2d-e45cc9d7c6e0b57b-00"}}, response = 2024/11/23-07:28:59.071989 140025230210624 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-21854/att-00: olev 3, work 47.909 s, result 6.268 ms, install 0.788 ms, input{raw 1.848 GiB zip 535.652 MiB}