2024/11/23-13:36:19.636309 139957231949376 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00: fork to child time = 0.008385 sec 2024/11/23-13:36:19.636923 139957231949376 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00 2024/11/23-13:36:19.643881 139957231949376 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00 2024/11/23-13:36:19.644744 139957231949376 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-13:36:19.645007 139957231949376 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-13:36:19.701229 139957231949376 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-26903/att-00/MANIFEST-206347 2024/11/23-13:36:19.717431 139957231949376 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-26903/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 261428, last_sequence is 3999067355, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 261169 2024/11/23-13:36:19.717437 139957231949376 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-13:36:19.717545 139957231949376 [version_set.cc:5721] Creating manifest 261428 2024/11/23-13:36:19.754153 139957231949376 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-13:36:19.775095 139957231949376 [compaction_job.cc:2437] [default] [JOB 26903] Compacting 1@2 + 15@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-13:36:19.775112 139957231949376 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 20148 Base level 2, inputs: [261425(27MB)], [261386(7851KB) 261355(36MB) 261356(34MB) 261357(34MB) 261358(34MB) 261359(36MB) 261360(35MB) 261361(34MB) 261362(35MB) 261363(34MB) 261387(35MB) 261388(35MB) 261389(35MB) 261390(33MB) 261391(35MB)] 2024/11/23-13:36:19.781479 139957231949376 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-13:36:19.781508 139957231949376 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-13:36:19.781526 139957231949376 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-13:36:19.781532 139957231949376 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-13:36:19.781542 139957231949376 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-13:36:20.133899 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261429.sst) /dev/shm/ToplingTemp/Topling-61349-2wIbp7 2024/11/23-13:36:21.039695 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261429.sst): first pass time = 0.91's, 148.183'MB/sec 2024/11/23-13:36:22.683038 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261429.sst): old prealloc_size = 147639500, real_size = 32136368 2024/11/23-13:36:22.683746 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261429.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 937780 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1080 GB zip-val = 0.0301 GB avg-val = 115.12 avg-zval = 32.10 2024/11/23-13:36:22.684710 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261429.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:36:24.184690 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261429.sst): second pass time = 1.64's, 65.734'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.50's, 24.308'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2603.982'MB/sec (index lex order gen) rebuild zvType time = 1.24's, 0.189'MB/sec write SST data time = 0.26's, 138.555'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 36489.698'MB/sec zip my value time = 1.64's, unzip length = 0.108'GB zip my value throughput = 65.734'MB/sec zip pipeline throughput = 65.737'MB/sec entries = 937780 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.12 avg-zval = 32.10 usrkeys = 937780 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.12 avg-zval = 32.10 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 937780 bytes = 3516688 } seq expand size = 6564460 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0188 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.4461 value = 3.5860 dict = 1.00 all = 3.6771 } Zip/UnZip{ index = 0.0957 value = 0.2789 dict = 1.00 all = 0.2720 } ---------------------------- total value len = 8.963144 GB avg = 0.168 KB (by entry num) total key len = 1.467079 GB avg = 0.028 KB total ukey len = 1.016634 GB avg = 0.015 KB total ukey num = 0.066488215 Billion total entry num = 0.053295944 Billion write speed all = 0.618198296 MB/sec (with seq num) write speed all = 0.592927533 MB/sec (without seq num) 2024/11/23-13:36:24.267541 139957231949376 [compaction_job.cc:1948] [default] [JOB 26903] Generated table #261429: 937780 keys, 36501266 bytes, temperature: kWarm 2024/11/23-13:36:24.267667 139957231949376 EVENT_LOG_v1 {"time_micros": 1732340184267592, "cf_name": "default", "job": 26903, "event": "table_file_creation", "file_number": 261429, "file_size": 36501266, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3999057421, "table_properties": {"data_size": 30106272, "index_size": 1795456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26257840, "raw_average_key_size": 28, "raw_value_size": 107959940, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 937780, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340180, "oldest_key_time": 0, "file_creation_time": 1732340180, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261429, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:36:24.345744 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261430.sst) /dev/shm/ToplingTemp/Topling-61349-slwQE8 2024/11/23-13:36:26.005480 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261430.sst): first pass time = 1.66's, 80.869'MB/sec 2024/11/23-13:36:27.800824 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261430.sst): old prealloc_size = 147639500, real_size = 32548656 2024/11/23-13:36:27.801433 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261430.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929506 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0305 GB avg-val = 116.40 avg-zval = 32.86 2024/11/23-13:36:27.802326 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261430.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:36:29.142912 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261430.sst): second pass time = 1.79's, 60.289'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.34's, 27.399'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3015.130'MB/sec (index lex order gen) rebuild zvType time = 1.11's, 0.209'MB/sec write SST data time = 0.23's, 161.613'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33589.544'MB/sec zip my value time = 1.79's, unzip length = 0.108'GB zip my value throughput = 60.289'MB/sec zip pipeline throughput = 60.289'MB/sec entries = 929506 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.40 avg-zval = 32.86 usrkeys = 929506 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.40 avg-zval = 32.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929506 bytes = 3369472 } seq expand size = 6506542 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5133 value = 3.5417 dict = 1.00 all = 3.6501 } Zip/UnZip{ index = 0.0951 value = 0.2823 dict = 1.00 all = 0.2740 } ---------------------------- total value len = 9.077842 GB avg = 0.167 KB (by entry num) total key len = 1.493105 GB avg = 0.028 KB total ukey len = 1.035224 GB avg = 0.015 KB total ukey num = 0.067417721 Billion total entry num = 0.054225450 Billion write speed all = 0.626354941 MB/sec (with seq num) write speed all = 0.600650998 MB/sec (without seq num) 2024/11/23-13:36:29.196337 139957231949376 [compaction_job.cc:1948] [default] [JOB 26903] Generated table #261430: 929506 keys, 36771000 bytes, temperature: kWarm 2024/11/23-13:36:29.196427 139957231949376 EVENT_LOG_v1 {"time_micros": 1732340189196379, "cf_name": "default", "job": 26903, "event": "table_file_creation", "file_number": 261430, "file_size": 36771000, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3564984029, "largest_seqno": 3999057260, "table_properties": {"data_size": 30547808, "index_size": 1768256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26026168, "raw_average_key_size": 28, "raw_value_size": 108191576, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929506, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340184, "oldest_key_time": 0, "file_creation_time": 1732340184, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261430, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:36:29.271046 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261431.sst) /dev/shm/ToplingTemp/Topling-61349-uMx2Z5 2024/11/23-13:36:30.957850 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261431.sst): first pass time = 1.69's, 79.572'MB/sec 2024/11/23-13:36:32.564803 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261431.sst): old prealloc_size = 147639500, real_size = 32818912 2024/11/23-13:36:32.565532 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261431.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931616 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 116.07 avg-zval = 33.08 2024/11/23-13:36:32.566417 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261431.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:36:33.689664 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261431.sst): second pass time = 1.61's, 67.370'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.12's, 32.832'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2512.549'MB/sec (index lex order gen) rebuild zvType time = 0.86's, 0.271'MB/sec write SST data time = 0.26's, 139.585'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 37592.117'MB/sec zip my value time = 1.61's, unzip length = 0.108'GB zip my value throughput = 67.370'MB/sec zip pipeline throughput = 67.370'MB/sec entries = 931616 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.07 avg-zval = 33.08 usrkeys = 931616 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.07 avg-zval = 33.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931616 bytes = 3260672 } seq expand size = 6521312 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5143 value = 3.5092 dict = 1.00 all = 3.6344 } Zip/UnZip{ index = 0.0951 value = 0.2850 dict = 1.00 all = 0.2752 } ---------------------------- total value len = 9.192496 GB avg = 0.167 KB (by entry num) total key len = 1.519190 GB avg = 0.028 KB total ukey len = 1.053856 GB avg = 0.015 KB total ukey num = 0.068349337 Billion total entry num = 0.055157066 Billion write speed all = 0.634523138 MB/sec (with seq num) write speed all = 0.608384632 MB/sec (without seq num) 2024/11/23-13:36:33.740668 139957231949376 [compaction_job.cc:1948] [default] [JOB 26903] Generated table #261431: 931616 keys, 36930262 bytes, temperature: kWarm 2024/11/23-13:36:33.740758 139957231949376 EVENT_LOG_v1 {"time_micros": 1732340193740710, "cf_name": "default", "job": 26903, "event": "table_file_creation", "file_number": 261431, "file_size": 36930262, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3801926538, "largest_seqno": 3999057290, "table_properties": {"data_size": 30813712, "index_size": 1772096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26085248, "raw_average_key_size": 28, "raw_value_size": 108132524, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931616, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340189, "oldest_key_time": 0, "file_creation_time": 1732340189, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261431, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:36:33.819683 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261432.sst) /dev/shm/ToplingTemp/Topling-61349-IAtCt8 2024/11/23-13:36:35.511958 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261432.sst): first pass time = 1.69's, 79.315'MB/sec 2024/11/23-13:36:37.114654 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261432.sst): old prealloc_size = 147639500, real_size = 34326976 2024/11/23-13:36:37.115435 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261432.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1086526 prefix = 4 raw-key = 0.0174 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1038 GB zip-val = 0.0320 GB avg-val = 95.53 avg-zval = 29.42 2024/11/23-13:36:37.115820 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261432.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0029 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:36:38.255316 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261432.sst): second pass time = 1.60's, 64.826'MB/sec, value only(77.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.14's, 31.653'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2774.765'MB/sec (index lex order gen) rebuild zvType time = 0.97's, 0.279'MB/sec write SST data time = 0.17's, 216.454'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.035'MB, throughput = 31511.661'MB/sec zip my value time = 1.60's, unzip length = 0.104'GB zip my value throughput = 64.826'MB/sec zip pipeline throughput = 64.826'MB/sec entries = 1086526 avg-key = 28.00 avg-zkey = 1.93 avg-val = 95.53 avg-zval = 29.42 usrkeys = 1086526 avg-key = 20.00 avg-zkey = 1.93 avg-val = 97.18 avg-zval = 29.42 TagRS{ kinds = RS_Key0_TagN bytes = 169880 } TagArray{ size = 256280 bytes = 832928 } seq expand size = 1793960 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0217 GB value = 0.1038 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0320 GB dict = 1.03 MB all = 0.0361 GB } UnZip/Zip{ index = 10.3790 value = 3.2475 dict = 1.00 all = 3.7175 } Zip/UnZip{ index = 0.0963 value = 0.3079 dict = 1.00 all = 0.2690 } ---------------------------- total value len = 9.298085 GB avg = 0.165 KB (by entry num) total key len = 1.549613 GB avg = 0.028 KB total ukey len = 1.075587 GB avg = 0.015 KB total ukey num = 0.069435863 Billion total entry num = 0.056243592 Billion write speed all = 0.642406261 MB/sec (with seq num) write speed all = 0.615760066 MB/sec (without seq num) 2024/11/23-13:36:38.334966 139957231949376 [compaction_job.cc:1948] [default] [JOB 26903] Generated table #261432: 1086526 keys, 36104232 bytes, temperature: kWarm 2024/11/23-13:36:38.335048 139957231949376 EVENT_LOG_v1 {"time_micros": 1732340198335013, "cf_name": "default", "job": 26903, "event": "table_file_creation", "file_number": 261432, "file_size": 36104232, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3999054939, "table_properties": {"data_size": 31961456, "index_size": 2093696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30422728, "raw_average_key_size": 28, "raw_value_size": 103795085, "raw_average_value_size": 95, "num_data_blocks": 1, "num_entries": 1086526, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340193, "oldest_key_time": 0, "file_creation_time": 1732340193, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261432, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:36:38.372803 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261433.sst) /dev/shm/ToplingTemp/Topling-61349-di7Vy5 2024/11/23-13:36:39.267545 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261433.sst): first pass time = 0.89's, 150.014'MB/sec 2024/11/23-13:36:41.017208 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261433.sst): old prealloc_size = 147639500, real_size = 36253232 2024/11/23-13:36:41.017764 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261433.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923917 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0343 GB avg-val = 117.27 avg-zval = 37.09 2024/11/23-13:36:41.018192 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261433.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:36:41.773235 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261433.sst): second pass time = 1.75's, 61.960'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.76's, 51.098'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3319.109'MB/sec (index lex order gen) rebuild zvType time = 0.63's, 0.367'MB/sec write SST data time = 0.13's, 305.087'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 37627.603'MB/sec zip my value time = 1.75's, unzip length = 0.108'GB zip my value throughput = 61.960'MB/sec zip pipeline throughput = 61.960'MB/sec entries = 923917 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.27 avg-zval = 37.09 usrkeys = 923917 avg-key = 20.00 avg-zkey = 1.90 avg-val = 120.04 avg-zval = 37.09 TagRS{ kinds = RS_Key0_TagN bytes = 144480 } TagArray{ size = 365760 bytes = 1371616 } seq expand size = 2560320 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0343 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.5136 value = 3.1621 dict = 1.00 all = 3.4744 } Zip/UnZip{ index = 0.0951 value = 0.3162 dict = 1.00 all = 0.2878 } ---------------------------- total value len = 9.408994 GB avg = 0.165 KB (by entry num) total key len = 1.575483 GB avg = 0.028 KB total ukey len = 1.094065 GB avg = 0.016 KB total ukey num = 0.070359780 Billion total entry num = 0.057167509 Billion write speed all = 0.650370842 MB/sec (with seq num) write speed all = 0.623292569 MB/sec (without seq num) 2024/11/23-13:36:41.821533 139957231949376 [compaction_job.cc:1948] [default] [JOB 26903] Generated table #261433: 923917 keys, 38630341 bytes, temperature: kWarm 2024/11/23-13:36:41.821609 139957231949376 EVENT_LOG_v1 {"time_micros": 1732340201821561, "cf_name": "default", "job": 26903, "event": "table_file_creation", "file_number": 261433, "file_size": 38630341, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3957058858, "table_properties": {"data_size": 34264480, "index_size": 1757568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25869676, "raw_average_key_size": 28, "raw_value_size": 108348186, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923917, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340198, "oldest_key_time": 0, "file_creation_time": 1732340198, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261433, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:36:41.858673 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261434.sst) /dev/shm/ToplingTemp/Topling-61349-K6cqB6 2024/11/23-13:36:42.725546 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261434.sst): first pass time = 0.87's, 154.838'MB/sec 2024/11/23-13:36:44.480646 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261434.sst): old prealloc_size = 147639500, real_size = 32600912 2024/11/23-13:36:44.481199 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261434.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931166 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0306 GB avg-val = 116.14 avg-zval = 32.86 2024/11/23-13:36:44.482028 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261434.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:36:45.550119 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261434.sst): second pass time = 1.75's, 61.640'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.07's, 34.436'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3354.094'MB/sec (index lex order gen) rebuild zvType time = 0.90's, 0.258'MB/sec write SST data time = 0.17's, 220.058'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 38459.382'MB/sec zip my value time = 1.75's, unzip length = 0.108'GB zip my value throughput = 61.640'MB/sec zip pipeline throughput = 105.229'MB/sec entries = 931166 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.14 avg-zval = 32.86 usrkeys = 931166 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.14 avg-zval = 32.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931166 bytes = 3375488 } seq expand size = 6518162 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5145 value = 3.5345 dict = 1.00 all = 3.6445 } Zip/UnZip{ index = 0.0951 value = 0.2829 dict = 1.00 all = 0.2744 } ---------------------------- total value len = 9.523657 GB avg = 0.164 KB (by entry num) total key len = 1.601555 GB avg = 0.028 KB total ukey len = 1.112688 GB avg = 0.016 KB total ukey num = 0.071290946 Billion total entry num = 0.058098675 Billion write speed all = 0.658556298 MB/sec (with seq num) write speed all = 0.631043116 MB/sec (without seq num) 2024/11/23-13:36:45.597378 139957231949376 [compaction_job.cc:1948] [default] [JOB 26903] Generated table #261434: 931166 keys, 36827934 bytes, temperature: kWarm 2024/11/23-13:36:45.597439 139957231949376 EVENT_LOG_v1 {"time_micros": 1732340205597407, "cf_name": "default", "job": 26903, "event": "table_file_creation", "file_number": 261434, "file_size": 36827934, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3487497056, "largest_seqno": 3999057389, "table_properties": {"data_size": 30596736, "index_size": 1771200, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26072648, "raw_average_key_size": 28, "raw_value_size": 108145220, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931166, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340201, "oldest_key_time": 0, "file_creation_time": 1732340201, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261434, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:36:45.633869 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261435.sst) /dev/shm/ToplingTemp/Topling-61349-fWMLf9 2024/11/23-13:36:46.519526 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261435.sst): first pass time = 0.89's, 151.553'MB/sec 2024/11/23-13:36:48.028144 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261435.sst): old prealloc_size = 147639500, real_size = 32698128 2024/11/23-13:36:48.028671 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261435.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931057 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0307 GB avg-val = 116.16 avg-zval = 32.97 2024/11/23-13:36:48.029482 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261435.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:36:49.086787 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261435.sst): second pass time = 1.51's, 71.726'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.06's, 34.770'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3506.540'MB/sec (index lex order gen) rebuild zvType time = 0.91's, 0.255'MB/sec write SST data time = 0.14's, 254.482'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35566.167'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.726'MB/sec zip pipeline throughput = 71.726'MB/sec entries = 931057 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.16 avg-zval = 32.97 usrkeys = 931057 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.16 avg-zval = 32.97 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931057 bytes = 3258720 } seq expand size = 6517399 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5122 value = 3.5235 dict = 1.00 all = 3.6464 } Zip/UnZip{ index = 0.0951 value = 0.2838 dict = 1.00 all = 0.2742 } ---------------------------- total value len = 9.638323 GB avg = 0.163 KB (by entry num) total key len = 1.627625 GB avg = 0.028 KB total ukey len = 1.131309 GB avg = 0.016 KB total ukey num = 0.072222003 Billion total entry num = 0.059029732 Billion write speed all = 0.666747522 MB/sec (with seq num) write speed all = 0.638799281 MB/sec (without seq num) 2024/11/23-13:36:49.132707 139957231949376 [compaction_job.cc:1948] [default] [JOB 26903] Generated table #261435: 931057 keys, 36808416 bytes, temperature: kWarm 2024/11/23-13:36:49.132775 139957231949376 EVENT_LOG_v1 {"time_micros": 1732340209132740, "cf_name": "default", "job": 26903, "event": "table_file_creation", "file_number": 261435, "file_size": 36808416, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3807172320, "largest_seqno": 3999057149, "table_properties": {"data_size": 30693760, "index_size": 1771392, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26069596, "raw_average_key_size": 28, "raw_value_size": 108148166, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931057, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340205, "oldest_key_time": 0, "file_creation_time": 1732340205, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261435, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:36:49.176810 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261436.sst) /dev/shm/ToplingTemp/Topling-61349-Hy8qq8 2024/11/23-13:36:50.066516 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261436.sst): first pass time = 0.89's, 150.864'MB/sec 2024/11/23-13:36:51.538632 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261436.sst): old prealloc_size = 147639500, real_size = 33334592 2024/11/23-13:36:51.539155 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261436.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929402 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0313 GB avg-val = 116.41 avg-zval = 33.71 2024/11/23-13:36:51.539963 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261436.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:36:52.589052 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261436.sst): second pass time = 1.47's, 73.541'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.05's, 35.865'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3506.156'MB/sec (index lex order gen) rebuild zvType time = 0.92's, 0.252'MB/sec write SST data time = 0.13's, 291.659'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 36056.681'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.541'MB/sec zip pipeline throughput = 73.541'MB/sec entries = 929402 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.41 avg-zval = 33.71 usrkeys = 929402 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.41 avg-zval = 33.71 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929402 bytes = 3485280 } seq expand size = 6505814 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5136 value = 3.4529 dict = 1.00 all = 3.5628 } Zip/UnZip{ index = 0.0951 value = 0.2896 dict = 1.00 all = 0.2807 } ---------------------------- total value len = 9.753023 GB avg = 0.163 KB (by entry num) total key len = 1.653648 GB avg = 0.028 KB total ukey len = 1.149897 GB avg = 0.016 KB total ukey num = 0.073151405 Billion total entry num = 0.059959134 Billion write speed all = 0.674936012 MB/sec (with seq num) write speed all = 0.646553619 MB/sec (without seq num) 2024/11/23-13:36:52.636499 139957231949376 [compaction_job.cc:1948] [default] [JOB 26903] Generated table #261436: 929402 keys, 37672248 bytes, temperature: kWarm 2024/11/23-13:36:52.636582 139957231949376 EVENT_LOG_v1 {"time_micros": 1732340212636534, "cf_name": "default", "job": 26903, "event": "table_file_creation", "file_number": 261436, "file_size": 37672248, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3999057441, "table_properties": {"data_size": 31334064, "index_size": 1768000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26023256, "raw_average_key_size": 28, "raw_value_size": 108194608, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929402, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340209, "oldest_key_time": 0, "file_creation_time": 1732340209, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261436, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:36:52.672011 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261437.sst) /dev/shm/ToplingTemp/Topling-61349-0GPf76 2024/11/23-13:36:53.619874 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261437.sst): first pass time = 0.95's, 141.607'MB/sec 2024/11/23-13:36:55.098446 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261437.sst): old prealloc_size = 147639500, real_size = 32631248 2024/11/23-13:36:55.098992 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261437.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928744 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0306 GB avg-val = 116.52 avg-zval = 32.98 2024/11/23-13:36:55.099830 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261437.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:36:56.175861 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261437.sst): second pass time = 1.48's, 73.223'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.08's, 34.204'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3369.063'MB/sec (index lex order gen) rebuild zvType time = 0.93's, 0.250'MB/sec write SST data time = 0.15's, 247.864'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34347.043'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.223'MB/sec zip pipeline throughput = 73.223'MB/sec entries = 928744 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.52 avg-zval = 32.98 usrkeys = 928744 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.52 avg-zval = 32.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928744 bytes = 3366704 } seq expand size = 6501208 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5122 value = 3.5327 dict = 1.00 all = 3.6421 } Zip/UnZip{ index = 0.0951 value = 0.2831 dict = 1.00 all = 0.2746 } ---------------------------- total value len = 9.867737 GB avg = 0.162 KB (by entry num) total key len = 1.679653 GB avg = 0.028 KB total ukey len = 1.168472 GB avg = 0.016 KB total ukey num = 0.074080149 Billion total entry num = 0.060887878 Billion write speed all = 0.683117419 MB/sec (with seq num) write speed all = 0.654301509 MB/sec (without seq num) 2024/11/23-13:36:56.222279 139957231949376 [compaction_job.cc:1948] [default] [JOB 26903] Generated table #261437: 928744 keys, 36851304 bytes, temperature: kWarm 2024/11/23-13:36:56.222348 139957231949376 EVENT_LOG_v1 {"time_micros": 1732340216222312, "cf_name": "default", "job": 26903, "event": "table_file_creation", "file_number": 261437, "file_size": 36851304, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3680980241, "largest_seqno": 3999056381, "table_properties": {"data_size": 30631872, "index_size": 1766976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26004832, "raw_average_key_size": 28, "raw_value_size": 108213012, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928744, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340212, "oldest_key_time": 0, "file_creation_time": 1732340212, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261437, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:36:56.261042 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261438.sst) /dev/shm/ToplingTemp/Topling-61349-rbiLY6 2024/11/23-13:36:57.448169 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261438.sst): first pass time = 1.19's, 113.065'MB/sec 2024/11/23-13:36:59.010517 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261438.sst): old prealloc_size = 147639500, real_size = 34857344 2024/11/23-13:36:59.011104 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261438.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925928 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0329 GB avg-val = 116.95 avg-zval = 35.48 2024/11/23-13:36:59.011942 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261438.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:36:59.861496 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261438.sst): second pass time = 1.56's, 69.363'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 46.047'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3147.887'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 309.941'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34818.141'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.363'MB/sec zip pipeline throughput = 69.363'MB/sec entries = 925928 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.95 avg-zval = 35.48 usrkeys = 925928 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.95 avg-zval = 35.48 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925928 bytes = 3472240 } seq expand size = 6481496 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0329 GB dict = 1.08 MB all = 0.0392 GB } UnZip/Zip{ index = 10.4441 value = 3.2963 dict = 1.00 all = 3.4253 } Zip/UnZip{ index = 0.0957 value = 0.3034 dict = 1.00 all = 0.2919 } ---------------------------- total value len = 9.982511 GB avg = 0.161 KB (by entry num) total key len = 1.705579 GB avg = 0.028 KB total ukey len = 1.186991 GB avg = 0.016 KB total ukey num = 0.075006077 Billion total entry num = 0.061813806 Billion write speed all = 0.691290150 MB/sec (with seq num) write speed all = 0.662042411 MB/sec (without seq num) 2024/11/23-13:36:59.910797 139957231949376 [compaction_job.cc:1948] [default] [JOB 26903] Generated table #261438: 925928 keys, 39184086 bytes, temperature: kWarm 2024/11/23-13:36:59.910867 139957231949376 EVENT_LOG_v1 {"time_micros": 1732340219910831, "cf_name": "default", "job": 26903, "event": "table_file_creation", "file_number": 261438, "file_size": 39184086, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369194706, "largest_seqno": 3999057120, "table_properties": {"data_size": 32852528, "index_size": 1773120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25925984, "raw_average_key_size": 28, "raw_value_size": 108291888, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925928, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340216, "oldest_key_time": 0, "file_creation_time": 1732340216, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261438, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:36:59.949187 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261439.sst) /dev/shm/ToplingTemp/Topling-61349-DpfoG8 2024/11/23-13:37:00.893374 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261439.sst): first pass time = 0.94's, 142.159'MB/sec 2024/11/23-13:37:02.214941 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261439.sst): old prealloc_size = 147639500, real_size = 32792496 2024/11/23-13:37:02.215546 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261439.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931008 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 116.16 avg-zval = 33.06 2024/11/23-13:37:02.216358 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261439.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:37:03.071696 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261439.sst): second pass time = 1.32's, 81.884'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.210'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3202.299'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.13's, 289.358'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32731.642'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.884'MB/sec zip pipeline throughput = 81.884'MB/sec entries = 931008 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.16 avg-zval = 33.06 usrkeys = 931008 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.16 avg-zval = 33.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931008 bytes = 3374912 } seq expand size = 6517056 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.4448 value = 3.5140 dict = 1.00 all = 3.6256 } Zip/UnZip{ index = 0.0957 value = 0.2846 dict = 1.00 all = 0.2758 } ---------------------------- total value len = 10.097177 GB avg = 0.161 KB (by entry num) total key len = 1.731647 GB avg = 0.028 KB total ukey len = 1.205611 GB avg = 0.016 KB total ukey num = 0.075937085 Billion total entry num = 0.062744814 Billion write speed all = 0.699481080 MB/sec (with seq num) write speed all = 0.669798462 MB/sec (without seq num) 2024/11/23-13:37:03.116748 139957231949376 [compaction_job.cc:1948] [default] [JOB 26903] Generated table #261439: 931008 keys, 37019200 bytes, temperature: kWarm 2024/11/23-13:37:03.116819 139957231949376 EVENT_LOG_v1 {"time_micros": 1732340223116784, "cf_name": "default", "job": 26903, "event": "table_file_creation", "file_number": 261439, "file_size": 37019200, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3610601794, "largest_seqno": 3999056554, "table_properties": {"data_size": 30776864, "index_size": 1782720, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26068224, "raw_average_key_size": 28, "raw_value_size": 108149544, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931008, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340219, "oldest_key_time": 0, "file_creation_time": 1732340219, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261439, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:37:03.151366 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261440.sst) /dev/shm/ToplingTemp/Topling-61349-GtSzU8 2024/11/23-13:37:04.080031 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261440.sst): first pass time = 0.93's, 144.534'MB/sec 2024/11/23-13:37:05.402384 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261440.sst): old prealloc_size = 147639500, real_size = 32480096 2024/11/23-13:37:05.402925 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261440.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931273 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0305 GB avg-val = 116.12 avg-zval = 32.72 2024/11/23-13:37:05.403767 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261440.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:37:06.249702 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261440.sst): second pass time = 1.32's, 81.822'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.323'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3416.860'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 293.040'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33996.028'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.822'MB/sec zip pipeline throughput = 81.822'MB/sec entries = 931273 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.12 avg-zval = 32.72 usrkeys = 931273 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.12 avg-zval = 32.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931273 bytes = 3375872 } seq expand size = 6518911 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5119 value = 3.5485 dict = 1.00 all = 3.6564 } Zip/UnZip{ index = 0.0951 value = 0.2818 dict = 1.00 all = 0.2735 } ---------------------------- total value len = 10.211838 GB avg = 0.160 KB (by entry num) total key len = 1.757723 GB avg = 0.028 KB total ukey len = 1.224236 GB avg = 0.016 KB total ukey num = 0.076868358 Billion total entry num = 0.063676087 Billion write speed all = 0.707670355 MB/sec (with seq num) write speed all = 0.677552841 MB/sec (without seq num) 2024/11/23-13:37:06.293990 139957231949376 [compaction_job.cc:1948] [default] [JOB 26903] Generated table #261440: 931273 keys, 36707396 bytes, temperature: kWarm 2024/11/23-13:37:06.294069 139957231949376 EVENT_LOG_v1 {"time_micros": 1732340226294031, "cf_name": "default", "job": 26903, "event": "table_file_creation", "file_number": 261440, "file_size": 36707396, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 3999056424, "table_properties": {"data_size": 30475280, "index_size": 1771840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26075644, "raw_average_key_size": 28, "raw_value_size": 108142090, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931273, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340223, "oldest_key_time": 0, "file_creation_time": 1732340223, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261440, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:37:06.330039 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261441.sst) /dev/shm/ToplingTemp/Topling-61349-IHsru9 2024/11/23-13:37:07.218643 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261441.sst): first pass time = 0.89's, 151.051'MB/sec 2024/11/23-13:37:08.520367 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261441.sst): old prealloc_size = 147639500, real_size = 33476384 2024/11/23-13:37:08.520919 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261441.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925498 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 117.02 avg-zval = 34.01 2024/11/23-13:37:08.521777 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261441.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:37:09.373824 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261441.sst): second pass time = 1.30's, 83.263'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.294'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3391.036'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 302.393'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 37297.045'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.263'MB/sec zip pipeline throughput = 83.263'MB/sec entries = 925498 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.02 avg-zval = 34.01 usrkeys = 925498 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.02 avg-zval = 34.01 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925498 bytes = 3470640 } seq expand size = 6478486 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.4467 value = 3.4412 dict = 1.00 all = 3.5505 } Zip/UnZip{ index = 0.0957 value = 0.2906 dict = 1.00 all = 0.2816 } ---------------------------- total value len = 10.326621 GB avg = 0.160 KB (by entry num) total key len = 1.783637 GB avg = 0.028 KB total ukey len = 1.242746 GB avg = 0.016 KB total ukey num = 0.077793856 Billion total entry num = 0.064601585 Billion write speed all = 0.715856450 MB/sec (with seq num) write speed all = 0.685306836 MB/sec (without seq num) 2024/11/23-13:37:09.421331 139957231949376 [compaction_job.cc:1948] [default] [JOB 26903] Generated table #261441: 925498 keys, 37801994 bytes, temperature: kWarm 2024/11/23-13:37:09.421400 139957231949376 EVENT_LOG_v1 {"time_micros": 1732340229421365, "cf_name": "default", "job": 26903, "event": "table_file_creation", "file_number": 261441, "file_size": 37801994, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 3999056437, "table_properties": {"data_size": 31472976, "index_size": 1771840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25913944, "raw_average_key_size": 28, "raw_value_size": 108303884, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925498, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340226, "oldest_key_time": 0, "file_creation_time": 1732340226, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261441, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:37:09.457075 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261442.sst) /dev/shm/ToplingTemp/Topling-61349-5QrBc8 2024/11/23-13:37:10.373903 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261442.sst): first pass time = 0.92's, 146.400'MB/sec 2024/11/23-13:37:11.696856 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261442.sst): old prealloc_size = 147639500, real_size = 33855136 2024/11/23-13:37:11.697423 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261442.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932992 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0318 GB avg-val = 115.86 avg-zval = 34.12 2024/11/23-13:37:11.698197 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261442.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:37:12.519396 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261442.sst): second pass time = 1.32's, 81.783'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 45.796'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3285.081'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 293.711'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34966.012'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.783'MB/sec zip pipeline throughput = 81.783'MB/sec entries = 932992 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.86 avg-zval = 34.12 usrkeys = 932992 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.48 avg-zval = 34.12 TagRS{ kinds = RS_Key0_TagN bytes = 145880 } TagArray{ size = 749995 bytes = 2812496 } seq expand size = 5249965 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.4449 value = 3.3954 dict = 1.00 all = 3.5632 } Zip/UnZip{ index = 0.0957 value = 0.2945 dict = 1.00 all = 0.2806 } ---------------------------- total value len = 10.439965 GB avg = 0.159 KB (by entry num) total key len = 1.809761 GB avg = 0.028 KB total ukey len = 1.261406 GB avg = 0.016 KB total ukey num = 0.078726848 Billion total entry num = 0.065534577 Billion write speed all = 0.723965993 MB/sec (with seq num) write speed all = 0.692980936 MB/sec (without seq num) 2024/11/23-13:37:12.566053 139957231949376 [compaction_job.cc:1948] [default] [JOB 26903] Generated table #261442: 932992 keys, 37667790 bytes, temperature: kWarm 2024/11/23-13:37:12.566124 139957231949376 EVENT_LOG_v1 {"time_micros": 1732340232566088, "cf_name": "default", "job": 26903, "event": "table_file_creation", "file_number": 261442, "file_size": 37667790, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3999056456, "table_properties": {"data_size": 31835216, "index_size": 1786496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26123776, "raw_average_key_size": 28, "raw_value_size": 108093980, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932992, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340229, "oldest_key_time": 0, "file_creation_time": 1732340229, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261442, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:37:12.610684 139957231949376 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261443.sst) /dev/shm/ToplingTemp/Topling-61349-W70go7 2024/11/23-13:37:12.842192 139957231949376 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261443.sst): first pass time = 0.23's, 235.723'MB/sec 2024/11/23-13:37:13.359369 139957231949376 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261443.sst): old prealloc_size = 147639500, real_size = 13748064 2024/11/23-13:37:13.359655 139957231949376 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261443.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 380671 prefix = 4 raw-key = 0.0061 GB zip-key = 0.0007 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0439 GB zip-val = 0.0129 GB avg-val = 115.33 avg-zval = 33.95 2024/11/23-13:37:13.360012 139957231949376 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261443.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0009 GB, waited 0.000 sec, Key+Value = 0.055 GB 2024/11/23-13:37:13.713806 139957231949376 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00/261443.sst): second pass time = 0.52's, 85.065'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.35's, 43.274'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2749.087'MB/sec (index lex order gen) rebuild zvType time = 0.30's, 0.316'MB/sec write SST data time = 0.05's, 291.288'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.445'MB, throughput = 15947.982'MB/sec zip my value time = 0.52's, unzip length = 0.044'GB zip my value throughput = 85.065'MB/sec zip pipeline throughput = 85.065'MB/sec entries = 380671 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.33 avg-zval = 33.95 usrkeys = 380671 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.33 avg-zval = 33.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 380671 bytes = 1237200 } seq expand size = 2664697 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0076 GB value = 0.0439 GB dict = 0.44 MB all = 0.0546 GB } __ZipSize{ index = 0.0007 GB value = 0.0129 GB dict = 0.44 MB all = 0.0153 GB } UnZip/Zip{ index = 10.4323 value = 3.3974 dict = 1.00 all = 3.5576 } Zip/UnZip{ index = 0.0959 value = 0.2943 dict = 1.00 all = 0.2811 } ---------------------------- total value len = 10.486533 GB avg = 0.159 KB (by entry num) total key len = 1.820419 GB avg = 0.028 KB total ukey len = 1.269020 GB avg = 0.016 KB total ukey num = 0.079107519 Billion total entry num = 0.065915248 Billion write speed all = 0.727296820 MB/sec (with seq num) write speed all = 0.696133979 MB/sec (without seq num) 2024/11/23-13:37:13.737156 139957231949376 [compaction_job.cc:1948] [default] [JOB 26903] Generated table #261443: 380671 keys, 15336917 bytes, temperature: kWarm 2024/11/23-13:37:13.737225 139957231949376 EVENT_LOG_v1 {"time_micros": 1732340233737190, "cf_name": "default", "job": 26903, "event": "table_file_creation", "file_number": 261443, "file_size": 15336917, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3951302061, "largest_seqno": 3997585778, "table_properties": {"data_size": 12922896, "index_size": 729792, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10658788, "raw_average_key_size": 28, "raw_value_size": 43903806, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 380671, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732340232, "oldest_key_time": 0, "file_creation_time": 1732340232, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261443, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:37:13.753670 139957231949376 [compaction_job.cc:703] job-26903: subcompact[0], size: 0.536814 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26903/att-00 [261429,261430,261431,261432,261433,261434,261435,261436,261437,261438,261439,261440,261441,261442,261443] 2024/11/23-13:37:13.847368 139957231949376 (Original Log Time 2024/11/23-13:36:19.775168) EVENT_LOG_v1 {"time_micros": 1732340179775127, "job": 26903, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [261425], "files_L3": [261386, 261355, 261356, 261357, 261358, 261359, 261360, 261361, 261362, 261363, 261387, 261388, 261389, 261390, 261391], "score": 0.993624, "input_data_size": 551156241, "oldest_snapshot_seqno": 3999067355} 2024/11/23-13:37:13.847371 139957231949376 (Original Log Time 2024/11/23-13:37:13.845182) [compaction_job.cc:2022] [default] [JOB 26903] Compacted 1@2 + 15@3 files to L3 => 536814386 bytes 2024/11/23-13:37:13.847373 139957231949376 (Original Log Time 2024/11/23-13:37:13.847301) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 10.2 rd, 9.9 wr, level 3, files in(1, 15) out(15 +0 blob) MB in(27.1, 498.5 +0.0 blob) out(511.9 +0.0 blob), read-write-amplify(38.3) write-amplify(18.9) OK, records in: 13600197, records dropped: 43113 output_compression: Snappy 2024/11/23-13:37:13.847380 139957231949376 (Original Log Time 2024/11/23-13:37:13.847347) EVENT_LOG_v1 {"time_micros": 1732340233847326, "cf": "default", "job": 26903, "event": "compaction_finished", "compaction_time_micros": 53978513, "compaction_time_cpu_micros": 23768242, "output_level": 3, "num_output_files": 15, "total_output_size": 536814386, "num_input_records": 13600197, "num_output_records": 13557084, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 432313992, "file_range_sync_nanos": 0, "file_fsync_nanos": 11659, "file_prepare_write_nanos": 47647, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/23-13:37:13.965782 139957231949376 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732340179,"executesMs":54202,"compactionJobId":26903,"attempt":0,"compactionInputRawBytes":1939733907,"compactionInputZipBytes":551156241,"compactionOutputRawBytes":0,"compactionOutputZipBytes":536814386,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-ee433171c54c1970f1ca636f1de12537-a1bbff022a63f99f-00"}}, response = 2024/11/23-13:37:13.965878 139957231949376 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-26903/att-00: olev 3, work 54.203 s, result 5.939 ms, install 2.220 ms, input{raw 1.807 GiB zip 525.624 MiB}