2024/11/24-00:53:50.381542 139957299091008 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00: fork to child time = 0.007763 sec 2024/11/24-00:53:50.382269 139957299091008 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00 2024/11/24-00:53:50.389440 139957299091008 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00 2024/11/24-00:53:50.390347 139957299091008 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-00:53:50.390606 139957299091008 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-00:53:50.439365 139957299091008 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-38921/att-00/MANIFEST-206347 2024/11/24-00:53:50.464840 139957299091008 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-38921/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 286428, last_sequence is 4359140602, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 286263 2024/11/24-00:53:50.464846 139957299091008 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-00:53:50.465013 139957299091008 [version_set.cc:5721] Creating manifest 286428 2024/11/24-00:53:50.508118 139957299091008 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-00:53:50.531573 139957299091008 [compaction_job.cc:2437] [default] [JOB 38921] Compacting 1@2 + 12@3 files to L3, score 0.65, subcompactions 1 : 1 2024/11/24-00:53:50.531586 139957299091008 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 29160 Base level 2, inputs: [286376(27MB)], [286411(35MB) 286412(1965KB) 286318(34MB) 286319(34MB) 286320(35MB) 286321(22MB) 286322(34MB) 286323(35MB) 286339(35MB) 286340(35MB) 286341(35MB) 286413(34MB)] 2024/11/24-00:53:50.538160 139957299091008 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-00:53:50.538201 139957299091008 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-00:53:50.538220 139957299091008 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-00:53:50.538225 139957299091008 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-00:53:50.538235 139957299091008 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-00:53:51.396490 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286429.sst) /dev/shm/ToplingTemp/Topling-231580-mm5j1b 2024/11/24-00:53:51.941853 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286429.sst): first pass time = 0.55's, 246.126'MB/sec 2024/11/24-00:53:53.284706 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286429.sst): old prealloc_size = 147639500, real_size = 33391904 2024/11/24-00:53:53.285327 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286429.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925470 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 117.03 avg-zval = 33.93 2024/11/24-00:53:53.286118 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286429.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:53:54.138432 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286429.sst): second pass time = 1.34's, 80.704'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.774'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2947.916'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 298.233'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33049.596'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.704'MB/sec zip pipeline throughput = 80.704'MB/sec entries = 925470 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.03 avg-zval = 33.93 usrkeys = 925470 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.03 avg-zval = 33.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925470 bytes = 3123472 } seq expand size = 6478290 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5129 value = 3.4492 dict = 1.00 all = 3.5915 } Zip/UnZip{ index = 0.0951 value = 0.2899 dict = 1.00 all = 0.2784 } ---------------------------- total value len = 8.963403 GB avg = 0.168 KB (by entry num) total key len = 1.466734 GB avg = 0.028 KB total ukey len = 1.016387 GB avg = 0.015 KB total ukey num = 0.066475905 Billion total entry num = 0.053283634 Billion write speed all = 0.181324550 MB/sec (with seq num) write speed all = 0.173914001 MB/sec (without seq num) 2024/11/24-00:53:54.183823 139957299091008 [compaction_job.cc:1948] [default] [JOB 38921] Generated table #286429: 925470 keys, 37370662 bytes, temperature: kWarm 2024/11/24-00:53:54.183918 139957299091008 EVENT_LOG_v1 {"time_micros": 1732380834183857, "cf_name": "default", "job": 38921, "event": "table_file_creation", "file_number": 286429, "file_size": 37370662, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178548, "largest_seqno": 4358701218, "table_properties": {"data_size": 31399696, "index_size": 1760640, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25913160, "raw_average_key_size": 28, "raw_value_size": 108304636, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925470, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732380831, "oldest_key_time": 0, "file_creation_time": 1732380831, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 286429, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:53:54.221714 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286430.sst) /dev/shm/ToplingTemp/Topling-231580-p3xDva 2024/11/24-00:53:55.211349 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286430.sst): first pass time = 0.99's, 135.628'MB/sec 2024/11/24-00:53:56.540327 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286430.sst): old prealloc_size = 147639500, real_size = 32862000 2024/11/24-00:53:56.540862 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286430.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928077 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.62 avg-zval = 33.26 2024/11/24-00:53:56.541697 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286430.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:53:57.391159 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286430.sst): second pass time = 1.33's, 81.490'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.309'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3474.849'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 292.567'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36767.140'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.490'MB/sec zip pipeline throughput = 81.490'MB/sec entries = 928077 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.62 avg-zval = 33.26 usrkeys = 928077 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.62 avg-zval = 33.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928077 bytes = 3132272 } seq expand size = 6496539 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5142 value = 3.5067 dict = 1.00 all = 3.6425 } Zip/UnZip{ index = 0.0951 value = 0.2852 dict = 1.00 all = 0.2745 } ---------------------------- total value len = 9.078131 GB avg = 0.167 KB (by entry num) total key len = 1.492720 GB avg = 0.028 KB total ukey len = 1.034949 GB avg = 0.015 KB total ukey num = 0.067403982 Billion total entry num = 0.054211711 Billion write speed all = 0.183760433 MB/sec (with seq num) write speed all = 0.176221235 MB/sec (without seq num) 2024/11/24-00:53:57.437143 139957299091008 [compaction_job.cc:1948] [default] [JOB 38921] Generated table #286430: 928077 keys, 36847600 bytes, temperature: kWarm 2024/11/24-00:53:57.437216 139957299091008 EVENT_LOG_v1 {"time_micros": 1732380837437172, "cf_name": "default", "job": 38921, "event": "table_file_creation", "file_number": 286430, "file_size": 36847600, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178559, "largest_seqno": 4358701043, "table_properties": {"data_size": 30864416, "index_size": 1765376, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25986156, "raw_average_key_size": 28, "raw_value_size": 108231678, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928077, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732380834, "oldest_key_time": 0, "file_creation_time": 1732380834, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 286430, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:53:57.480648 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286431.sst) /dev/shm/ToplingTemp/Topling-231580-3Q1gic 2024/11/24-00:53:58.326453 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286431.sst): first pass time = 0.85's, 158.693'MB/sec 2024/11/24-00:53:59.670719 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286431.sst): old prealloc_size = 147639500, real_size = 31984544 2024/11/24-00:53:59.671271 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286431.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928762 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0300 GB avg-val = 116.51 avg-zval = 32.29 2024/11/24-00:53:59.672026 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286431.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:54:00.506057 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286431.sst): second pass time = 1.34's, 80.585'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.064'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3333.045'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.328'MB/sec write SST data time = 0.13's, 281.441'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 29215.339'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.585'MB/sec zip pipeline throughput = 80.585'MB/sec entries = 928762 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.51 avg-zval = 32.29 usrkeys = 928762 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.51 avg-zval = 32.29 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928762 bytes = 3134592 } seq expand size = 6501334 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.5136 value = 3.6088 dict = 1.00 all = 3.7311 } Zip/UnZip{ index = 0.0951 value = 0.2771 dict = 1.00 all = 0.2680 } ---------------------------- total value len = 9.192845 GB avg = 0.167 KB (by entry num) total key len = 1.518726 GB avg = 0.028 KB total ukey len = 1.053524 GB avg = 0.015 KB total ukey num = 0.068332744 Billion total entry num = 0.055140473 Billion write speed all = 0.186196570 MB/sec (with seq num) write speed all = 0.178528624 MB/sec (without seq num) 2024/11/24-00:54:00.550737 139957299091008 [compaction_job.cc:1948] [default] [JOB 38921] Generated table #286431: 928762 keys, 35972288 bytes, temperature: kWarm 2024/11/24-00:54:00.550809 139957299091008 EVENT_LOG_v1 {"time_micros": 1732380840550771, "cf_name": "default", "job": 38921, "event": "table_file_creation", "file_number": 286431, "file_size": 35972288, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178575, "largest_seqno": 4358701058, "table_properties": {"data_size": 29985360, "index_size": 1766784, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26005336, "raw_average_key_size": 28, "raw_value_size": 108212508, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928762, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732380837, "oldest_key_time": 0, "file_creation_time": 1732380837, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 286431, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:54:00.587957 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286432.sst) /dev/shm/ToplingTemp/Topling-231580-Kmz9Ra 2024/11/24-00:54:01.543071 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286432.sst): first pass time = 0.96's, 140.533'MB/sec 2024/11/24-00:54:02.902677 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286432.sst): old prealloc_size = 147639500, real_size = 32166992 2024/11/24-00:54:02.903262 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286432.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925863 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0302 GB avg-val = 116.97 avg-zval = 32.59 2024/11/24-00:54:02.904020 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286432.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:54:03.728938 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286432.sst): second pass time = 1.36's, 79.729'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.749'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3146.994'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.332'MB/sec write SST data time = 0.13's, 283.169'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33953.627'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.729'MB/sec zip pipeline throughput = 79.729'MB/sec entries = 925863 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.97 avg-zval = 32.59 usrkeys = 925863 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.97 avg-zval = 32.59 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925863 bytes = 3124800 } seq expand size = 6481041 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.5131 value = 3.5890 dict = 1.00 all = 3.7131 } Zip/UnZip{ index = 0.0951 value = 0.2786 dict = 1.00 all = 0.2693 } ---------------------------- total value len = 9.307619 GB avg = 0.166 KB (by entry num) total key len = 1.544650 GB avg = 0.028 KB total ukey len = 1.072041 GB avg = 0.015 KB total ukey num = 0.069258607 Billion total entry num = 0.056066336 Billion write speed all = 0.188631735 MB/sec (with seq num) write speed all = 0.180835475 MB/sec (without seq num) 2024/11/24-00:54:03.773380 139957299091008 [compaction_job.cc:1948] [default] [JOB 38921] Generated table #286432: 925863 keys, 36146902 bytes, temperature: kWarm 2024/11/24-00:54:03.773451 139957299091008 EVENT_LOG_v1 {"time_micros": 1732380843773416, "cf_name": "default", "job": 38921, "event": "table_file_creation", "file_number": 286432, "file_size": 36146902, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178589, "largest_seqno": 4358701073, "table_properties": {"data_size": 30174016, "index_size": 1761344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25924164, "raw_average_key_size": 28, "raw_value_size": 108293670, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925863, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732380840, "oldest_key_time": 0, "file_creation_time": 1732380840, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 286432, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:54:03.812633 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286433.sst) /dev/shm/ToplingTemp/Topling-231580-n3yJzb 2024/11/24-00:54:04.977106 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286433.sst): first pass time = 1.16's, 87.968'MB/sec 2024/11/24-00:54:05.970159 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286433.sst): old prealloc_size = 147639500, real_size = 24425632 2024/11/24-00:54:05.970590 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286433.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 707844 prefix = 4 raw-key = 0.0113 GB zip-key = 0.0014 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0826 GB zip-val = 0.0229 GB avg-val = 116.71 avg-zval = 32.34 2024/11/24-00:54:05.971199 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286433.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0018 GB, waited 0.000 sec, Key+Value = 0.102 GB 2024/11/24-00:54:06.619656 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286433.sst): second pass time = 0.99's, 83.254'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 42.296'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3421.992'MB/sec (index lex order gen) rebuild zvType time = 0.55's, 0.320'MB/sec write SST data time = 0.10's, 283.991'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.830'MB, throughput = 23361.192'MB/sec zip my value time = 0.99's, unzip length = 0.083'GB zip my value throughput = 83.254'MB/sec zip pipeline throughput = 83.254'MB/sec entries = 707844 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.71 avg-zval = 32.34 usrkeys = 707844 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.71 avg-zval = 32.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 707844 bytes = 2388992 } seq expand size = 4954908 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0142 GB value = 0.0826 GB dict = 0.83 MB all = 0.1024 GB } __ZipSize{ index = 0.0014 GB value = 0.0229 GB dict = 0.83 MB all = 0.0275 GB } UnZip/Zip{ index = 10.4419 value = 3.6087 dict = 1.00 all = 3.7288 } Zip/UnZip{ index = 0.0958 value = 0.2771 dict = 1.00 all = 0.2682 } ---------------------------- total value len = 9.395188 GB avg = 0.165 KB (by entry num) total key len = 1.564470 GB avg = 0.028 KB total ukey len = 1.086198 GB avg = 0.016 KB total ukey num = 0.069966451 Billion total entry num = 0.056774180 Billion write speed all = 0.190488759 MB/sec (with seq num) write speed all = 0.182594466 MB/sec (without seq num) 2024/11/24-00:54:06.656314 139957299091008 [compaction_job.cc:1948] [default] [JOB 38921] Generated table #286433: 707844 keys, 27470602 bytes, temperature: kWarm 2024/11/24-00:54:06.656385 139957299091008 EVENT_LOG_v1 {"time_micros": 1732380846656349, "cf_name": "default", "job": 38921, "event": "table_file_creation", "file_number": 286433, "file_size": 27470602, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4278801260, "largest_seqno": 4358701088, "table_properties": {"data_size": 22892688, "index_size": 1355776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 19819632, "raw_average_key_size": 28, "raw_value_size": 82613500, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 707844, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732380843, "oldest_key_time": 0, "file_creation_time": 1732380843, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 286433, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:54:06.689032 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286434.sst) /dev/shm/ToplingTemp/Topling-231580-FCYRZc 2024/11/24-00:54:07.229909 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286434.sst): first pass time = 0.54's, 248.166'MB/sec 2024/11/24-00:54:08.557440 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286434.sst): old prealloc_size = 147639500, real_size = 32167200 2024/11/24-00:54:08.557952 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286434.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925594 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0302 GB avg-val = 117.01 avg-zval = 32.60 2024/11/24-00:54:08.558756 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286434.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:54:09.384773 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286434.sst): second pass time = 1.33's, 81.664'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.691'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3624.555'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 286.715'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 23119.635'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.664'MB/sec zip pipeline throughput = 81.664'MB/sec entries = 925594 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.01 avg-zval = 32.60 usrkeys = 925594 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.01 avg-zval = 32.60 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925594 bytes = 3123888 } seq expand size = 6479158 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.5128 value = 3.5891 dict = 1.00 all = 3.7132 } Zip/UnZip{ index = 0.0951 value = 0.2786 dict = 1.00 all = 0.2693 } ---------------------------- total value len = 9.509968 GB avg = 0.165 KB (by entry num) total key len = 1.590386 GB avg = 0.028 KB total ukey len = 1.104710 GB avg = 0.016 KB total ukey num = 0.070892045 Billion total entry num = 0.057699774 Billion write speed all = 0.192924927 MB/sec (with seq num) write speed all = 0.184902318 MB/sec (without seq num) 2024/11/24-00:54:09.431118 139957299091008 [compaction_job.cc:1948] [default] [JOB 38921] Generated table #286434: 925594 keys, 36146146 bytes, temperature: kWarm 2024/11/24-00:54:09.431203 139957299091008 EVENT_LOG_v1 {"time_micros": 1732380849431151, "cf_name": "default", "job": 38921, "event": "table_file_creation", "file_number": 286434, "file_size": 36146146, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178645, "largest_seqno": 4358700699, "table_properties": {"data_size": 30174736, "index_size": 1760896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25916632, "raw_average_key_size": 28, "raw_value_size": 108301108, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925594, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732380846, "oldest_key_time": 0, "file_creation_time": 1732380846, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 286434, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:54:09.470705 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286435.sst) /dev/shm/ToplingTemp/Topling-231580-pIlPac 2024/11/24-00:54:10.413559 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286435.sst): first pass time = 0.94's, 142.358'MB/sec 2024/11/24-00:54:11.759831 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286435.sst): old prealloc_size = 147639500, real_size = 32959760 2024/11/24-00:54:11.760390 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286435.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925390 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 117.04 avg-zval = 33.46 2024/11/24-00:54:11.761131 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286435.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:54:12.595777 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286435.sst): second pass time = 1.35's, 80.519'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.188'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3279.917'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.12's, 296.510'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 27323.792'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.519'MB/sec zip pipeline throughput = 80.519'MB/sec entries = 925390 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.04 avg-zval = 33.46 usrkeys = 925390 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.04 avg-zval = 33.46 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925390 bytes = 3123200 } seq expand size = 6477730 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5120 value = 3.4974 dict = 1.00 all = 3.6336 } Zip/UnZip{ index = 0.0951 value = 0.2859 dict = 1.00 all = 0.2752 } ---------------------------- total value len = 9.624753 GB avg = 0.164 KB (by entry num) total key len = 1.616297 GB avg = 0.028 KB total ukey len = 1.123218 GB avg = 0.016 KB total ukey num = 0.071817435 Billion total entry num = 0.058625164 Billion write speed all = 0.195359322 MB/sec (with seq num) write speed all = 0.187208502 MB/sec (without seq num) 2024/11/24-00:54:12.641792 139957299091008 [compaction_job.cc:1948] [default] [JOB 38921] Generated table #286435: 925390 keys, 36938062 bytes, temperature: kWarm 2024/11/24-00:54:12.641861 139957299091008 EVENT_LOG_v1 {"time_micros": 1732380852641826, "cf_name": "default", "job": 38921, "event": "table_file_creation", "file_number": 286435, "file_size": 36938062, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178675, "largest_seqno": 4358700939, "table_properties": {"data_size": 30967616, "index_size": 1760640, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25910920, "raw_average_key_size": 28, "raw_value_size": 108306884, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925390, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732380849, "oldest_key_time": 0, "file_creation_time": 1732380849, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 286435, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:54:12.683426 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286436.sst) /dev/shm/ToplingTemp/Topling-231580-J2VBqc 2024/11/24-00:54:13.719147 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286436.sst): first pass time = 1.04's, 129.593'MB/sec 2024/11/24-00:54:15.063727 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286436.sst): old prealloc_size = 147639500, real_size = 33461008 2024/11/24-00:54:15.064285 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286436.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925566 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 117.01 avg-zval = 34.00 2024/11/24-00:54:15.065036 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286436.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:54:15.916162 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286436.sst): second pass time = 1.34's, 80.623'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.923'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3297.675'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 292.467'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32212.644'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.623'MB/sec zip pipeline throughput = 80.623'MB/sec entries = 925566 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.01 avg-zval = 34.00 usrkeys = 925566 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.01 avg-zval = 34.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925566 bytes = 3123808 } seq expand size = 6478962 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5124 value = 3.4416 dict = 1.00 all = 3.5849 } Zip/UnZip{ index = 0.0951 value = 0.2906 dict = 1.00 all = 0.2789 } ---------------------------- total value len = 9.739534 GB avg = 0.164 KB (by entry num) total key len = 1.642213 GB avg = 0.028 KB total ukey len = 1.141729 GB avg = 0.016 KB total ukey num = 0.072743001 Billion total entry num = 0.059550730 Billion write speed all = 0.197793091 MB/sec (with seq num) write speed all = 0.189514064 MB/sec (without seq num) 2024/11/24-00:54:15.961219 139957299091008 [compaction_job.cc:1948] [default] [JOB 38921] Generated table #286436: 925566 keys, 37439950 bytes, temperature: kWarm 2024/11/24-00:54:15.961290 139957299091008 EVENT_LOG_v1 {"time_micros": 1732380855961254, "cf_name": "default", "job": 38921, "event": "table_file_creation", "file_number": 286436, "file_size": 37439950, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178686, "largest_seqno": 4358701277, "table_properties": {"data_size": 31468544, "index_size": 1760896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25915848, "raw_average_key_size": 28, "raw_value_size": 108301924, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925566, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732380852, "oldest_key_time": 0, "file_creation_time": 1732380852, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 286436, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:54:15.997300 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286437.sst) /dev/shm/ToplingTemp/Topling-231580-cRQB19 2024/11/24-00:54:17.026563 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286437.sst): first pass time = 1.03's, 130.407'MB/sec 2024/11/24-00:54:18.380434 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286437.sst): old prealloc_size = 147639500, real_size = 33591440 2024/11/24-00:54:18.380997 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286437.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925551 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 117.01 avg-zval = 34.14 2024/11/24-00:54:18.382772 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286437.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:54:19.239091 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286437.sst): second pass time = 1.35's, 80.044'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.756'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3259.788'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 293.851'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32171.876'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.044'MB/sec zip pipeline throughput = 80.044'MB/sec entries = 925551 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.01 avg-zval = 34.14 usrkeys = 925551 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.01 avg-zval = 34.14 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925551 bytes = 3123744 } seq expand size = 6478857 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5130 value = 3.4274 dict = 1.00 all = 3.5724 } Zip/UnZip{ index = 0.0951 value = 0.2918 dict = 1.00 all = 0.2799 } ---------------------------- total value len = 9.854315 GB avg = 0.163 KB (by entry num) total key len = 1.668128 GB avg = 0.028 KB total ukey len = 1.160240 GB avg = 0.016 KB total ukey num = 0.073668552 Billion total entry num = 0.060476281 Billion write speed all = 0.200226570 MB/sec (with seq num) write speed all = 0.191819354 MB/sec (without seq num) 2024/11/24-00:54:19.284204 139957299091008 [compaction_job.cc:1948] [default] [JOB 38921] Generated table #286437: 925551 keys, 37570398 bytes, temperature: kWarm 2024/11/24-00:54:19.284271 139957299091008 EVENT_LOG_v1 {"time_micros": 1732380859284238, "cf_name": "default", "job": 38921, "event": "table_file_creation", "file_number": 286437, "file_size": 37570398, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178700, "largest_seqno": 4358701378, "table_properties": {"data_size": 31599104, "index_size": 1760768, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25915428, "raw_average_key_size": 28, "raw_value_size": 108302418, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925551, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732380855, "oldest_key_time": 0, "file_creation_time": 1732380855, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 286437, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:54:19.320420 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286438.sst) /dev/shm/ToplingTemp/Topling-231580-jCfgka 2024/11/24-00:54:20.158071 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286438.sst): first pass time = 0.84's, 160.239'MB/sec 2024/11/24-00:54:21.519644 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286438.sst): old prealloc_size = 147639500, real_size = 33314704 2024/11/24-00:54:21.520221 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286438.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925746 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.98 avg-zval = 33.82 2024/11/24-00:54:21.520957 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286438.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:54:22.383175 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286438.sst): second pass time = 1.36's, 79.584'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.189'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3156.364'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.316'MB/sec write SST data time = 0.13's, 284.208'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32604.672'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.584'MB/sec zip pipeline throughput = 79.584'MB/sec entries = 925746 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.98 avg-zval = 33.82 usrkeys = 925746 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.98 avg-zval = 33.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925746 bytes = 3124400 } seq expand size = 6480222 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4439 value = 3.4588 dict = 1.00 all = 3.5989 } Zip/UnZip{ index = 0.0957 value = 0.2891 dict = 1.00 all = 0.2779 } ---------------------------- total value len = 9.969092 GB avg = 0.162 KB (by entry num) total key len = 1.694049 GB avg = 0.028 KB total ukey len = 1.178755 GB avg = 0.016 KB total ukey num = 0.074594298 Billion total entry num = 0.061402027 Billion write speed all = 0.202660419 MB/sec (with seq num) write speed all = 0.194124976 MB/sec (without seq num) 2024/11/24-00:54:22.429010 139957299091008 [compaction_job.cc:1948] [default] [JOB 38921] Generated table #286438: 925746 keys, 37293650 bytes, temperature: kWarm 2024/11/24-00:54:22.429084 139957299091008 EVENT_LOG_v1 {"time_micros": 1732380862429044, "cf_name": "default", "job": 38921, "event": "table_file_creation", "file_number": 286438, "file_size": 37293650, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178719, "largest_seqno": 4358700769, "table_properties": {"data_size": 31310272, "index_size": 1772800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25920888, "raw_average_key_size": 28, "raw_value_size": 108296892, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925746, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732380859, "oldest_key_time": 0, "file_creation_time": 1732380859, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 286438, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:54:22.471084 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286439.sst) /dev/shm/ToplingTemp/Topling-231580-ciFRhb 2024/11/24-00:54:23.485326 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286439.sst): first pass time = 1.01's, 132.338'MB/sec 2024/11/24-00:54:24.849128 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286439.sst): old prealloc_size = 147639500, real_size = 32619600 2024/11/24-00:54:24.849674 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286439.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928814 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0306 GB avg-val = 116.50 avg-zval = 32.95 2024/11/24-00:54:24.851474 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286439.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:54:25.703084 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286439.sst): second pass time = 1.36's, 79.391'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 42.869'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3400.418'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 282.027'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33942.951'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.391'MB/sec zip pipeline throughput = 79.391'MB/sec entries = 928814 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.50 avg-zval = 32.95 usrkeys = 928814 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.50 avg-zval = 32.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928814 bytes = 3134768 } seq expand size = 6501698 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.4453 value = 3.5353 dict = 1.00 all = 3.6665 } Zip/UnZip{ index = 0.0957 value = 0.2829 dict = 1.00 all = 0.2727 } ---------------------------- total value len = 10.083805 GB avg = 0.162 KB (by entry num) total key len = 1.720056 GB avg = 0.028 KB total ukey len = 1.197332 GB avg = 0.016 KB total ukey num = 0.075523112 Billion total entry num = 0.062330841 Billion write speed all = 0.205093750 MB/sec (with seq num) write speed all = 0.196429693 MB/sec (without seq num) 2024/11/24-00:54:25.748290 139957299091008 [compaction_job.cc:1948] [default] [JOB 38921] Generated table #286439: 928814 keys, 36606916 bytes, temperature: kWarm 2024/11/24-00:54:25.748361 139957299091008 EVENT_LOG_v1 {"time_micros": 1732380865748323, "cf_name": "default", "job": 38921, "event": "table_file_creation", "file_number": 286439, "file_size": 36606916, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178730, "largest_seqno": 4358700786, "table_properties": {"data_size": 30608768, "index_size": 1778432, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26006792, "raw_average_key_size": 28, "raw_value_size": 108210988, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928814, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732380862, "oldest_key_time": 0, "file_creation_time": 1732380862, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 286439, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:54:25.786205 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286440.sst) /dev/shm/ToplingTemp/Topling-231580-Y6nbAb 2024/11/24-00:54:25.824611 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286440.sst): first pass time = 0.04's, 225.874'MB/sec 2024/11/24-00:54:25.902074 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286440.sst): old prealloc_size = 147639500, real_size = 2559296 2024/11/24-00:54:25.902165 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286440.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 61039 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0001 GB avg-key = 16.00 avg-zkey = 1.95 raw-val = 0.0070 GB zip-val = 0.0024 GB avg-val = 114.00 avg-zval = 39.72 2024/11/24-00:54:25.902262 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286440.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0001 GB, waited 0.000 sec, Key+Value = 0.009 GB 2024/11/24-00:54:25.954333 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00/286440.sst): second pass time = 0.08's, 90.532'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.05's, 53.578'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1787.189'MB/sec (index lex order gen) rebuild zvType time = 0.04's, 0.350'MB/sec write SST data time = 0.01's, 324.617'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.071'MB, throughput = 2826.633'MB/sec zip my value time = 0.08's, unzip length = 0.007'GB zip my value throughput = 90.532'MB/sec zip pipeline throughput = 90.532'MB/sec entries = 61039 avg-key = 28.00 avg-zkey = 1.95 avg-val = 114.00 avg-zval = 39.72 usrkeys = 61039 avg-key = 20.00 avg-zkey = 1.95 avg-val = 121.00 avg-zval = 39.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 61039 bytes = 183136 } seq expand size = 427273 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0012 GB value = 0.0070 GB dict = 0.07 MB all = 0.0087 GB } __ZipSize{ index = 0.0001 GB value = 0.0024 GB dict = 0.07 MB all = 0.0028 GB } UnZip/Zip{ index = 10.2442 value = 2.8698 dict = 1.00 all = 3.0957 } Zip/UnZip{ index = 0.0976 value = 0.3485 dict = 1.00 all = 0.3230 } ---------------------------- total value len = 10.091190 GB avg = 0.162 KB (by entry num) total key len = 1.721765 GB avg = 0.028 KB total ukey len = 1.198552 GB avg = 0.016 KB total ukey num = 0.075584151 Billion total entry num = 0.062391880 Billion write speed all = 0.205250878 MB/sec (with seq num) write speed all = 0.196578374 MB/sec (without seq num) 2024/11/24-00:54:25.965213 139957299091008 [compaction_job.cc:1948] [default] [JOB 38921] Generated table #286440: 61039 keys, 2799842 bytes, temperature: kWarm 2024/11/24-00:54:25.965261 139957299091008 EVENT_LOG_v1 {"time_micros": 1732380865965235, "cf_name": "default", "job": 38921, "event": "table_file_creation", "file_number": 286440, "file_size": 2799842, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4342298208, "largest_seqno": 4351610623, "table_properties": {"data_size": 2424688, "index_size": 119168, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 1709092, "raw_average_key_size": 28, "raw_value_size": 6958446, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 61039, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732380865, "oldest_key_time": 0, "file_creation_time": 1732380865, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 286440, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:54:25.974111 139957299091008 [compaction_job.cc:703] job-38921: subcompact[0], size: 0.398603 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38921/att-00 [286429,286430,286431,286432,286433,286434,286435,286436,286437,286438,286439,286440] 2024/11/24-00:54:26.043210 139957299091008 (Original Log Time 2024/11/24-00:53:50.531637) EVENT_LOG_v1 {"time_micros": 1732380830531597, "job": 38921, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [286376], "files_L3": [286411, 286412, 286318, 286319, 286320, 286321, 286322, 286323, 286339, 286340, 286341, 286413], "score": 0.650076, "input_data_size": 422720708, "oldest_snapshot_seqno": 4359140503} 2024/11/24-00:54:26.043214 139957299091008 (Original Log Time 2024/11/24-00:54:26.042153) [compaction_job.cc:2022] [default] [JOB 38921] Compacted 1@2 + 12@3 files to L3 => 398603018 bytes 2024/11/24-00:54:26.043216 139957299091008 (Original Log Time 2024/11/24-00:54:26.043138) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 12 0 0 0] max score 0.01, MB/sec: 11.9 rd, 11.2 wr, level 3, files in(1, 12) out(12 +0 blob) MB in(27.1, 376.0 +0.0 blob) out(380.1 +0.0 blob), read-write-amplify(28.9) write-amplify(14.0) OK, records in: 10147125, records dropped: 113409 output_compression: Snappy 2024/11/24-00:54:26.043221 139957299091008 (Original Log Time 2024/11/24-00:54:26.043190) EVENT_LOG_v1 {"time_micros": 1732380866043160, "cf": "default", "job": 38921, "event": "compaction_finished", "compaction_time_micros": 35442480, "compaction_time_cpu_micros": 15104427, "output_level": 3, "num_output_files": 12, "total_output_size": 398603018, "num_input_records": 10147125, "num_output_records": 10033716, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 273978399, "file_range_sync_nanos": 0, "file_fsync_nanos": 8171, "file_prepare_write_nanos": 29963, "lsm_state": [0, 0, 0, 12, 0, 0, 0]} 2024/11/24-00:54:26.173481 139957299091008 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732380830,"executesMs":35652,"compactionJobId":38921,"attempt":0,"compactionInputRawBytes":1469382778,"compactionInputZipBytes":422720708,"compactionOutputRawBytes":0,"compactionOutputZipBytes":398603018,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-3be3529f7dffaf22aeafb1e78cc15dbd-f40651fe4c7ad940-00"}}, response = 2024/11/24-00:54:26.173562 139957299091008 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-38921/att-00: olev 3, work 35.653 s, result 7.657 ms, install 1.083 ms, input{raw 1.368 GiB zip 403.138 MiB}