2024/11/23-12:02:31.529599 139957383018048 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00: fork to child time = 0.007681 sec 2024/11/23-12:02:31.530206 139957383018048 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00 2024/11/23-12:02:31.536956 139957383018048 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00 2024/11/23-12:02:31.537850 139957383018048 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-12:02:31.538116 139957383018048 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-12:02:31.596302 139957383018048 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-25666/att-00/MANIFEST-206347 2024/11/23-12:02:31.628620 139957383018048 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-25666/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 258753, last_sequence is 3964903874, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 258556 2024/11/23-12:02:31.628627 139957383018048 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-12:02:31.628749 139957383018048 [version_set.cc:5721] Creating manifest 258753 2024/11/23-12:02:31.673209 139957383018048 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-12:02:31.704184 139957383018048 [compaction_job.cc:2437] [default] [JOB 25666] Compacting 1@2 + 13@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-12:02:31.704212 139957383018048 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 19231 Base level 2, inputs: [258701(27MB)], [258750(35MB) 258751(10MB) 258642(37MB) 258643(33MB) 258664(36MB) 258665(35MB) 258666(35MB) 258667(36MB) 258668(35MB) 258669(36MB) 258670(35MB) 258671(35MB) 258739(35MB)] 2024/11/23-12:02:31.713221 139957383018048 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-12:02:31.713262 139957383018048 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-12:02:31.713289 139957383018048 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-12:02:31.713298 139957383018048 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-12:02:31.713315 139957383018048 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-12:02:32.279498 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258754.sst) /dev/shm/ToplingTemp/Topling-43573-sOt0w0 2024/11/23-12:02:32.822550 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258754.sst): first pass time = 0.54's, 247.177'MB/sec 2024/11/23-12:02:34.909253 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258754.sst): old prealloc_size = 147639500, real_size = 33266000 2024/11/23-12:02:34.909768 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258754.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929017 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0313 GB avg-val = 116.47 avg-zval = 33.64 2024/11/23-12:02:34.910530 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258754.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-12:02:35.726768 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258754.sst): second pass time = 2.09's, 51.874'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 45.052'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3564.447'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.337'MB/sec write SST data time = 0.13's, 289.038'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34739.968'MB/sec zip my value time = 2.09's, unzip length = 0.108'GB zip my value throughput = 51.874'MB/sec zip pipeline throughput = 152.184'MB/sec entries = 929017 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.47 avg-zval = 33.64 usrkeys = 929017 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.62 avg-zval = 33.64 TagRS{ kinds = RS_Key0_TagN bytes = 145240 } TagArray{ size = 683192 bytes = 2561984 } seq expand size = 4782344 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.4446 value = 3.4621 dict = 1.00 all = 3.6442 } Zip/UnZip{ index = 0.0957 value = 0.2888 dict = 1.00 all = 0.2744 } ---------------------------- total value len = 8.961608 GB avg = 0.168 KB (by entry num) total key len = 1.466834 GB avg = 0.028 KB total ukey len = 1.016458 GB avg = 0.015 KB total ukey num = 0.066479452 Billion total entry num = 0.053287181 Billion write speed all = 0.927507433 MB/sec (with seq num) write speed all = 0.889592461 MB/sec (without seq num) 2024/11/23-12:02:35.773449 139957383018048 [compaction_job.cc:1948] [default] [JOB 25666] Generated table #258754: 929017 keys, 36830094 bytes, temperature: kWarm 2024/11/23-12:02:35.773532 139957383018048 EVENT_LOG_v1 {"time_micros": 1732334555773479, "cf_name": "default", "job": 25666, "event": "table_file_creation", "file_number": 258754, "file_size": 36830094, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3964689591, "table_properties": {"data_size": 31254592, "index_size": 1778944, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26012476, "raw_average_key_size": 28, "raw_value_size": 108205366, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929017, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732334552, "oldest_key_time": 0, "file_creation_time": 1732334552, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 258754, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-12:02:35.812414 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258755.sst) /dev/shm/ToplingTemp/Topling-43573-1mcQd1 2024/11/23-12:02:36.926815 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258755.sst): first pass time = 1.11's, 120.445'MB/sec 2024/11/23-12:02:38.449734 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258755.sst): old prealloc_size = 147639500, real_size = 33229504 2024/11/23-12:02:38.450399 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258755.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931394 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0312 GB avg-val = 116.10 avg-zval = 33.51 2024/11/23-12:02:38.451320 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258755.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-12:02:39.325925 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258755.sst): second pass time = 1.52's, 71.062'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 42.884'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2768.428'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.15's, 256.738'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36570.828'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 71.062'MB/sec zip pipeline throughput = 102.963'MB/sec entries = 931394 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.10 avg-zval = 33.51 usrkeys = 931394 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.10 avg-zval = 33.51 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931394 bytes = 3492736 } seq expand size = 6519758 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.4446 value = 3.4645 dict = 1.00 all = 3.5721 } Zip/UnZip{ index = 0.0957 value = 0.2886 dict = 1.00 all = 0.2799 } ---------------------------- total value len = 9.076266 GB avg = 0.167 KB (by entry num) total key len = 1.492913 GB avg = 0.028 KB total ukey len = 1.035086 GB avg = 0.015 KB total ukey num = 0.067410846 Billion total entry num = 0.054218575 Billion write speed all = 0.939723837 MB/sec (with seq num) write speed all = 0.901158503 MB/sec (without seq num) 2024/11/23-12:02:39.371865 139957383018048 [compaction_job.cc:1948] [default] [JOB 25666] Generated table #258755: 931394 keys, 37573676 bytes, temperature: kWarm 2024/11/23-12:02:39.371934 139957383018048 EVENT_LOG_v1 {"time_micros": 1732334559371899, "cf_name": "default", "job": 25666, "event": "table_file_creation", "file_number": 258755, "file_size": 37573676, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3964689945, "table_properties": {"data_size": 31212976, "index_size": 1783488, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26079032, "raw_average_key_size": 28, "raw_value_size": 108138720, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931394, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732334555, "oldest_key_time": 0, "file_creation_time": 1732334555, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 258755, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-12:02:39.406447 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258756.sst) /dev/shm/ToplingTemp/Topling-43573-dtIN60 2024/11/23-12:02:40.216956 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258756.sst): first pass time = 0.81's, 165.605'MB/sec 2024/11/23-12:02:41.776442 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258756.sst): old prealloc_size = 147639500, real_size = 31602128 2024/11/23-12:02:41.777036 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258756.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1085 GB zip-val = 0.0296 GB avg-val = 118.00 avg-zval = 32.20 2024/11/23-12:02:41.777840 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258756.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-12:02:42.599171 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258756.sst): second pass time = 1.56's, 69.597'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.646'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3083.780'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 284.101'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 33670.866'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.597'MB/sec zip pipeline throughput = 130.262'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 32.20 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 32.20 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3447392 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.09 MB all = 0.0359 GB } UnZip/Zip{ index = 10.3760 value = 3.6648 dict = 1.00 all = 3.7378 } Zip/UnZip{ index = 0.0964 value = 0.2729 dict = 1.00 all = 0.2675 } ---------------------------- total value len = 9.191178 GB avg = 0.167 KB (by entry num) total key len = 1.518653 GB avg = 0.028 KB total ukey len = 1.053472 GB avg = 0.015 KB total ukey num = 0.068330146 Billion total entry num = 0.055137875 Billion write speed all = 0.951952482 MB/sec (with seq num) write speed all = 0.912744667 MB/sec (without seq num) 2024/11/23-12:02:42.644395 139957383018048 [compaction_job.cc:1948] [default] [JOB 25666] Generated table #258756: 919300 keys, 35908658 bytes, temperature: kWarm 2024/11/23-12:02:42.644471 139957383018048 EVENT_LOG_v1 {"time_micros": 1732334562644435, "cf_name": "default", "job": 25666, "event": "table_file_creation", "file_number": 258756, "file_size": 35908658, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273946337, "largest_seqno": 3956117597, "table_properties": {"data_size": 29600128, "index_size": 1771968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732334559, "oldest_key_time": 0, "file_creation_time": 1732334559, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 258756, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-12:02:42.688260 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258757.sst) /dev/shm/ToplingTemp/Topling-43573-XugwLX 2024/11/23-12:02:43.586184 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258757.sst): first pass time = 0.90's, 149.483'MB/sec 2024/11/23-12:02:45.115413 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258757.sst): old prealloc_size = 147639500, real_size = 32308896 2024/11/23-12:02:45.116011 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258757.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 936072 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1080 GB zip-val = 0.0303 GB avg-val = 115.38 avg-zval = 32.35 2024/11/23-12:02:45.116833 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258757.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-12:02:45.977724 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258757.sst): second pass time = 1.53's, 70.685'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.389'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3179.612'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 287.266'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 34393.524'MB/sec zip my value time = 1.53's, unzip length = 0.108'GB zip my value throughput = 70.685'MB/sec zip pipeline throughput = 134.530'MB/sec entries = 936072 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.38 avg-zval = 32.35 usrkeys = 936072 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.38 avg-zval = 32.35 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 936072 bytes = 3393280 } seq expand size = 6552504 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.4439 value = 3.5667 dict = 1.00 all = 3.6720 } Zip/UnZip{ index = 0.0957 value = 0.2804 dict = 1.00 all = 0.2723 } ---------------------------- total value len = 9.305739 GB avg = 0.166 KB (by entry num) total key len = 1.544863 GB avg = 0.028 KB total ukey len = 1.072194 GB avg = 0.015 KB total ukey num = 0.069266218 Billion total entry num = 0.056073947 Billion write speed all = 0.964175424 MB/sec (with seq num) write speed all = 0.924313951 MB/sec (without seq num) 2024/11/23-12:02:46.040094 139957383018048 [compaction_job.cc:1948] [default] [JOB 25666] Generated table #258757: 936072 keys, 36551230 bytes, temperature: kWarm 2024/11/23-12:02:46.040161 139957383018048 EVENT_LOG_v1 {"time_micros": 1732334566040126, "cf_name": "default", "job": 25666, "event": "table_file_creation", "file_number": 258757, "file_size": 36551230, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3680980241, "largest_seqno": 3964690009, "table_properties": {"data_size": 30282128, "index_size": 1792576, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26210016, "raw_average_key_size": 28, "raw_value_size": 108007796, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 936072, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732334562, "oldest_key_time": 0, "file_creation_time": 1732334562, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 258757, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-12:02:46.076835 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258758.sst) /dev/shm/ToplingTemp/Topling-43573-4cUUEY 2024/11/23-12:02:46.938851 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258758.sst): first pass time = 0.86's, 155.709'MB/sec 2024/11/23-12:02:48.490298 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258758.sst): old prealloc_size = 147639500, real_size = 31501216 2024/11/23-12:02:48.490882 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258758.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927739 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0295 GB avg-val = 116.67 avg-zval = 31.79 2024/11/23-12:02:48.491684 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258758.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-12:02:49.318581 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258758.sst): second pass time = 1.55's, 69.805'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.265'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3169.614'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.13's, 284.041'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 28955.077'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 69.805'MB/sec zip pipeline throughput = 123.630'MB/sec entries = 927739 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.67 avg-zval = 31.79 usrkeys = 927739 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.67 avg-zval = 31.79 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927739 bytes = 3479040 } seq expand size = 6494173 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.08 MB all = 0.0358 GB } UnZip/Zip{ index = 10.4426 value = 3.6702 dict = 1.00 all = 3.7455 } Zip/UnZip{ index = 0.0958 value = 0.2725 dict = 1.00 all = 0.2670 } ---------------------------- total value len = 9.420474 GB avg = 0.165 KB (by entry num) total key len = 1.570840 GB avg = 0.028 KB total ukey len = 1.090748 GB avg = 0.016 KB total ukey num = 0.070193957 Billion total entry num = 0.057001686 Billion write speed all = 0.976389114 MB/sec (with seq num) write speed all = 0.935880162 MB/sec (without seq num) 2024/11/23-12:02:49.362970 139957383018048 [compaction_job.cc:1948] [default] [JOB 25666] Generated table #258758: 927739 keys, 35834512 bytes, temperature: kWarm 2024/11/23-12:02:49.363051 139957383018048 EVENT_LOG_v1 {"time_micros": 1732334569363015, "cf_name": "default", "job": 25666, "event": "table_file_creation", "file_number": 258758, "file_size": 35834512, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369194706, "largest_seqno": 3964690094, "table_properties": {"data_size": 29492240, "index_size": 1776832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25976692, "raw_average_key_size": 28, "raw_value_size": 108241086, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927739, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732334566, "oldest_key_time": 0, "file_creation_time": 1732334566, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 258758, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-12:02:49.402176 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258759.sst) /dev/shm/ToplingTemp/Topling-43573-6n5lF0 2024/11/23-12:02:50.278335 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258759.sst): first pass time = 0.88's, 153.195'MB/sec 2024/11/23-12:02:51.881650 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258759.sst): old prealloc_size = 147639500, real_size = 31578576 2024/11/23-12:02:51.882228 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258759.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931558 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0296 GB avg-val = 116.08 avg-zval = 31.75 2024/11/23-12:02:51.883064 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258759.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-12:02:52.721920 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258759.sst): second pass time = 1.60's, 67.489'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.614'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3176.170'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 279.971'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34589.643'MB/sec zip my value time = 1.60's, unzip length = 0.108'GB zip my value throughput = 67.489'MB/sec zip pipeline throughput = 114.492'MB/sec entries = 931558 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.08 avg-zval = 31.75 usrkeys = 931558 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.08 avg-zval = 31.75 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931558 bytes = 3376912 } seq expand size = 6520906 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.08 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5132 value = 3.6565 dict = 1.00 all = 3.7484 } Zip/UnZip{ index = 0.0951 value = 0.2735 dict = 1.00 all = 0.2668 } ---------------------------- total value len = 9.535129 GB avg = 0.165 KB (by entry num) total key len = 1.596923 GB avg = 0.028 KB total ukey len = 1.109380 GB avg = 0.016 KB total ukey num = 0.071125515 Billion total entry num = 0.057933244 Billion write speed all = 0.988592442 MB/sec (with seq num) write speed all = 0.947433910 MB/sec (without seq num) 2024/11/23-12:02:52.779493 139957383018048 [compaction_job.cc:1948] [default] [JOB 25666] Generated table #258759: 931558 keys, 35806490 bytes, temperature: kWarm 2024/11/23-12:02:52.779562 139957383018048 EVENT_LOG_v1 {"time_micros": 1732334572779526, "cf_name": "default", "job": 25666, "event": "table_file_creation", "file_number": 258759, "file_size": 35806490, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 3964689674, "table_properties": {"data_size": 29573312, "index_size": 1772160, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26083624, "raw_average_key_size": 28, "raw_value_size": 108134112, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931558, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732334569, "oldest_key_time": 0, "file_creation_time": 1732334569, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 258759, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-12:02:52.814289 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258760.sst) /dev/shm/ToplingTemp/Topling-43573-QS9HE0 2024/11/23-12:02:53.786476 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258760.sst): first pass time = 0.97's, 138.063'MB/sec 2024/11/23-12:02:55.350161 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258760.sst): old prealloc_size = 147639500, real_size = 32508944 2024/11/23-12:02:55.350689 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258760.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930437 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0305 GB avg-val = 116.25 avg-zval = 32.77 2024/11/23-12:02:55.351493 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258760.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-12:02:56.200586 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258760.sst): second pass time = 1.56's, 69.209'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.332'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3521.383'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 291.891'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 38094.576'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.209'MB/sec zip pipeline throughput = 118.772'MB/sec entries = 930437 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.25 avg-zval = 32.77 usrkeys = 930437 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.25 avg-zval = 32.77 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930437 bytes = 3489152 } seq expand size = 6513059 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4440 value = 3.5471 dict = 1.00 all = 3.6423 } Zip/UnZip{ index = 0.0957 value = 0.2819 dict = 1.00 all = 0.2746 } ---------------------------- total value len = 9.649808 GB avg = 0.164 KB (by entry num) total key len = 1.622976 GB avg = 0.028 KB total ukey len = 1.127988 GB avg = 0.016 KB total ukey num = 0.072055952 Billion total entry num = 0.058863681 Billion write speed all = 1.000781016 MB/sec (with seq num) write speed all = 0.958974373 MB/sec (without seq num) 2024/11/23-12:02:56.249564 139957383018048 [compaction_job.cc:1948] [default] [JOB 25666] Generated table #258760: 930437 keys, 36850030 bytes, temperature: kWarm 2024/11/23-12:02:56.249631 139957383018048 EVENT_LOG_v1 {"time_micros": 1732334576249596, "cf_name": "default", "job": 25666, "event": "table_file_creation", "file_number": 258760, "file_size": 36850030, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 3964689702, "table_properties": {"data_size": 30494400, "index_size": 1781760, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26052236, "raw_average_key_size": 28, "raw_value_size": 108165626, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930437, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732334572, "oldest_key_time": 0, "file_creation_time": 1732334572, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 258760, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-12:02:56.287087 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258761.sst) /dev/shm/ToplingTemp/Topling-43573-VZSZ5W 2024/11/23-12:02:57.244312 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258761.sst): first pass time = 0.96's, 140.221'MB/sec 2024/11/23-12:02:58.812351 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258761.sst): old prealloc_size = 147639500, real_size = 30323632 2024/11/23-12:02:58.812876 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258761.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930693 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0283 GB avg-val = 116.21 avg-zval = 30.42 2024/11/23-12:02:58.813686 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258761.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-12:02:59.640085 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258761.sst): second pass time = 1.57's, 69.011'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 41.738'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3532.536'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.332'MB/sec write SST data time = 0.13's, 272.468'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 30017.705'MB/sec zip my value time = 1.57's, unzip length = 0.108'GB zip my value throughput = 69.011'MB/sec zip pipeline throughput = 120.393'MB/sec entries = 930693 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.21 avg-zval = 30.42 usrkeys = 930693 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.21 avg-zval = 30.42 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930693 bytes = 3373776 } seq expand size = 6514851 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0283 GB dict = 1.08 MB all = 0.0345 GB } UnZip/Zip{ index = 10.4454 value = 3.8207 dict = 1.00 all = 3.8851 } Zip/UnZip{ index = 0.0957 value = 0.2617 dict = 1.00 all = 0.2574 } ---------------------------- total value len = 9.764481 GB avg = 0.163 KB (by entry num) total key len = 1.649035 GB avg = 0.028 KB total ukey len = 1.146602 GB avg = 0.016 KB total ukey num = 0.072986645 Billion total entry num = 0.059794374 Billion write speed all = 1.012965744 MB/sec (with seq num) write speed all = 0.970511060 MB/sec (without seq num) 2024/11/23-12:02:59.683545 139957383018048 [compaction_job.cc:1948] [default] [JOB 25666] Generated table #258761: 930693 keys, 34547048 bytes, temperature: kWarm 2024/11/23-12:02:59.683611 139957383018048 EVENT_LOG_v1 {"time_micros": 1732334579683578, "cf_name": "default", "job": 25666, "event": "table_file_creation", "file_number": 258761, "file_size": 34547048, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3674424249, "largest_seqno": 3964689721, "table_properties": {"data_size": 28308768, "index_size": 1782016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26059404, "raw_average_key_size": 28, "raw_value_size": 108158442, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930693, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732334576, "oldest_key_time": 0, "file_creation_time": 1732334576, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 258761, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-12:02:59.717553 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258762.sst) /dev/shm/ToplingTemp/Topling-43573-ovucNY 2024/11/23-12:03:00.715566 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258762.sst): first pass time = 1.00's, 134.491'MB/sec 2024/11/23-12:03:02.146224 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258762.sst): old prealloc_size = 147639500, real_size = 35382224 2024/11/23-12:03:02.146773 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258762.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929729 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0334 GB avg-val = 116.36 avg-zval = 35.89 2024/11/23-12:03:02.147519 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258762.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-12:03:02.968230 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258762.sst): second pass time = 1.43's, 75.664'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 47.670'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3394.020'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 301.012'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35518.678'MB/sec zip my value time = 1.43's, unzip length = 0.108'GB zip my value throughput = 75.664'MB/sec zip pipeline throughput = 75.664'MB/sec entries = 929729 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.36 avg-zval = 35.89 usrkeys = 929729 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.98 avg-zval = 35.89 TagRS{ kinds = RS_Key0_TagN bytes = 145360 } TagArray{ size = 746732 bytes = 2800256 } seq expand size = 5227124 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0334 GB dict = 1.08 MB all = 0.0392 GB } UnZip/Zip{ index = 10.4455 value = 3.2420 dict = 1.00 all = 3.4253 } Zip/UnZip{ index = 0.0957 value = 0.3084 dict = 1.00 all = 0.2919 } ---------------------------- total value len = 9.877894 GB avg = 0.163 KB (by entry num) total key len = 1.675067 GB avg = 0.028 KB total ukey len = 1.165197 GB avg = 0.016 KB total ukey num = 0.073916374 Billion total entry num = 0.060724103 Billion write speed all = 1.025038912 MB/sec (with seq num) write speed all = 0.981936840 MB/sec (without seq num) 2024/11/23-12:03:03.019158 139957383018048 [compaction_job.cc:1948] [default] [JOB 25666] Generated table #258762: 929729 keys, 39184204 bytes, temperature: kWarm 2024/11/23-12:03:03.019228 139957383018048 EVENT_LOG_v1 {"time_micros": 1732334583019193, "cf_name": "default", "job": 25666, "event": "table_file_creation", "file_number": 258762, "file_size": 39184204, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3964689325, "table_properties": {"data_size": 33369472, "index_size": 1780160, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26032412, "raw_average_key_size": 28, "raw_value_size": 108185450, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929729, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732334579, "oldest_key_time": 0, "file_creation_time": 1732334579, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 258762, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-12:03:03.059173 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258763.sst) /dev/shm/ToplingTemp/Topling-43573-W8qY00 2024/11/23-12:03:04.083082 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258763.sst): first pass time = 1.02's, 131.089'MB/sec 2024/11/23-12:03:05.458957 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258763.sst): old prealloc_size = 147639500, real_size = 30575648 2024/11/23-12:03:05.459526 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258763.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928157 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0286 GB avg-val = 116.61 avg-zval = 30.76 2024/11/23-12:03:05.461523 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258763.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-12:03:06.279700 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258763.sst): second pass time = 1.37's, 78.723'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.392'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3318.594'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 272.280'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36986.299'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 78.723'MB/sec zip pipeline throughput = 78.723'MB/sec entries = 928157 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.61 avg-zval = 30.76 usrkeys = 928157 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.61 avg-zval = 30.76 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928157 bytes = 3364592 } seq expand size = 6497099 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0286 GB dict = 1.08 MB all = 0.0348 GB } UnZip/Zip{ index = 10.3767 value = 3.7903 dict = 1.00 all = 3.8577 } Zip/UnZip{ index = 0.0964 value = 0.2638 dict = 1.00 all = 0.2592 } ---------------------------- total value len = 9.992620 GB avg = 0.162 KB (by entry num) total key len = 1.701056 GB avg = 0.028 KB total ukey len = 1.183760 GB avg = 0.016 KB total ukey num = 0.074844531 Billion total entry num = 0.061652260 Billion write speed all = 1.037219120 MB/sec (with seq num) write speed all = 0.993471095 MB/sec (without seq num) 2024/11/23-12:03:06.325201 139957383018048 [compaction_job.cc:1948] [default] [JOB 25666] Generated table #258763: 928157 keys, 34791922 bytes, temperature: kWarm 2024/11/23-12:03:06.325273 139957383018048 EVENT_LOG_v1 {"time_micros": 1732334586325235, "cf_name": "default", "job": 25666, "event": "table_file_creation", "file_number": 258763, "file_size": 34791922, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3532696339, "largest_seqno": 3964688898, "table_properties": {"data_size": 28554512, "index_size": 1788928, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25988396, "raw_average_key_size": 28, "raw_value_size": 108229354, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928157, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732334583, "oldest_key_time": 0, "file_creation_time": 1732334583, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 258763, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-12:03:06.364250 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258764.sst) /dev/shm/ToplingTemp/Topling-43573-d9xWw0 2024/11/23-12:03:07.383722 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258764.sst): first pass time = 1.02's, 131.659'MB/sec 2024/11/23-12:03:08.815453 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258764.sst): old prealloc_size = 147639500, real_size = 31213152 2024/11/23-12:03:08.816005 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258764.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931151 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0292 GB avg-val = 116.14 avg-zval = 31.36 2024/11/23-12:03:08.818007 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258764.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-12:03:09.649319 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258764.sst): second pass time = 1.43's, 75.576'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.501'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3366.990'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 275.635'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31739.169'MB/sec zip my value time = 1.43's, unzip length = 0.108'GB zip my value throughput = 75.576'MB/sec zip pipeline throughput = 75.576'MB/sec entries = 931151 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.14 avg-zval = 31.36 usrkeys = 931151 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.14 avg-zval = 31.36 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931151 bytes = 3375440 } seq expand size = 6518057 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4445 value = 3.7040 dict = 1.00 all = 3.7872 } Zip/UnZip{ index = 0.0957 value = 0.2700 dict = 1.00 all = 0.2640 } ---------------------------- total value len = 10.107284 GB avg = 0.162 KB (by entry num) total key len = 1.727128 GB avg = 0.028 KB total ukey len = 1.202383 GB avg = 0.016 KB total ukey num = 0.075775682 Billion total entry num = 0.062583411 Billion write speed all = 1.049388624 MB/sec (with seq num) write speed all = 1.004993129 MB/sec (without seq num) 2024/11/23-12:03:09.692833 139957383018048 [compaction_job.cc:1948] [default] [JOB 25666] Generated table #258764: 931151 keys, 35439986 bytes, temperature: kWarm 2024/11/23-12:03:09.692902 139957383018048 EVENT_LOG_v1 {"time_micros": 1732334589692867, "cf_name": "default", "job": 25666, "event": "table_file_creation", "file_number": 258764, "file_size": 35439986, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3604582272, "largest_seqno": 3964689923, "table_properties": {"data_size": 29197136, "index_size": 1783040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26072228, "raw_average_key_size": 28, "raw_value_size": 108145530, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931151, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732334586, "oldest_key_time": 0, "file_creation_time": 1732334586, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 258764, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-12:03:09.729252 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258765.sst) /dev/shm/ToplingTemp/Topling-43573-n3S36X 2024/11/23-12:03:10.780331 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258765.sst): first pass time = 1.05's, 127.701'MB/sec 2024/11/23-12:03:12.185159 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258765.sst): old prealloc_size = 147639500, real_size = 30774368 2024/11/23-12:03:12.185689 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258765.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931056 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0288 GB avg-val = 116.16 avg-zval = 30.89 2024/11/23-12:03:12.187262 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258765.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-12:03:13.031963 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258765.sst): second pass time = 1.40's, 77.026'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 41.472'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3494.759'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 278.587'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35387.352'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.026'MB/sec zip pipeline throughput = 77.026'MB/sec entries = 931056 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.16 avg-zval = 30.89 usrkeys = 931056 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.16 avg-zval = 30.89 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931056 bytes = 3491472 } seq expand size = 6517392 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0288 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.4446 value = 3.7606 dict = 1.00 all = 3.8220 } Zip/UnZip{ index = 0.0957 value = 0.2659 dict = 1.00 all = 0.2616 } ---------------------------- total value len = 10.221949 GB avg = 0.161 KB (by entry num) total key len = 1.753198 GB avg = 0.028 KB total ukey len = 1.221004 GB avg = 0.016 KB total ukey num = 0.076706738 Billion total entry num = 0.063514467 Billion write speed all = 1.061549573 MB/sec (with seq num) write speed all = 1.016507116 MB/sec (without seq num) 2024/11/23-12:03:13.076780 139957383018048 [compaction_job.cc:1948] [default] [JOB 25666] Generated table #258765: 931056 keys, 35117152 bytes, temperature: kWarm 2024/11/23-12:03:13.076846 139957383018048 EVENT_LOG_v1 {"time_micros": 1732334593076812, "cf_name": "default", "job": 25666, "event": "table_file_creation", "file_number": 258765, "file_size": 35117152, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 3964689524, "table_properties": {"data_size": 28758544, "index_size": 1782848, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26069568, "raw_average_key_size": 28, "raw_value_size": 108148200, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931056, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732334589, "oldest_key_time": 0, "file_creation_time": 1732334589, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 258765, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-12:03:13.111436 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258766.sst) /dev/shm/ToplingTemp/Topling-43573-mVjO0Y 2024/11/23-12:03:13.376805 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258766.sst): first pass time = 0.27's, 245.287'MB/sec 2024/11/23-12:03:14.018298 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258766.sst): old prealloc_size = 147639500, real_size = 16717824 2024/11/23-12:03:14.018627 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258766.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 448869 prefix = 4 raw-key = 0.0072 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0525 GB zip-val = 0.0157 GB avg-val = 116.99 avg-zval = 35.07 2024/11/23-12:03:14.019068 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258766.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.065 GB 2024/11/23-12:03:14.434820 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00/258766.sst): second pass time = 0.64's, 81.955'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.42's, 45.189'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2839.705'MB/sec (index lex order gen) rebuild zvType time = 0.35's, 0.317'MB/sec write SST data time = 0.06's, 303.352'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.530'MB, throughput = 18536.357'MB/sec zip my value time = 0.64's, unzip length = 0.053'GB zip my value throughput = 81.955'MB/sec zip pipeline throughput = 81.955'MB/sec entries = 448869 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.99 avg-zval = 35.07 usrkeys = 448869 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.99 avg-zval = 35.07 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 448869 bytes = 1683280 } seq expand size = 3142083 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0090 GB value = 0.0525 GB dict = 0.53 MB all = 0.0651 GB } __ZipSize{ index = 0.0009 GB value = 0.0157 GB dict = 0.53 MB all = 0.0188 GB } UnZip/Zip{ index = 10.3690 value = 3.3364 dict = 1.00 all = 3.4579 } Zip/UnZip{ index = 0.0964 value = 0.2997 dict = 1.00 all = 0.2892 } ---------------------------- total value len = 10.277605 GB avg = 0.161 KB (by entry num) total key len = 1.765766 GB avg = 0.028 KB total ukey len = 1.229981 GB avg = 0.016 KB total ukey num = 0.077155607 Billion total entry num = 0.063963336 Billion write speed all = 1.067464611 MB/sec (with seq num) write speed all = 1.022109470 MB/sec (without seq num) 2024/11/23-12:03:14.461529 139957383018048 [compaction_job.cc:1948] [default] [JOB 25666] Generated table #258766: 448869 keys, 18821203 bytes, temperature: kWarm 2024/11/23-12:03:14.461596 139957383018048 EVENT_LOG_v1 {"time_micros": 1732334594461561, "cf_name": "default", "job": 25666, "event": "table_file_creation", "file_number": 258766, "file_size": 18821203, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3416313872, "largest_seqno": 3964688939, "table_properties": {"data_size": 15739632, "index_size": 865792, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 12568332, "raw_average_key_size": 28, "raw_value_size": 52513550, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 448869, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732334593, "oldest_key_time": 0, "file_creation_time": 1732334593, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 258766, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-12:03:14.480635 139957383018048 [compaction_job.cc:703] job-25666: subcompact[0], size: 0.453256 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25666/att-00 [258754,258755,258756,258757,258758,258759,258760,258761,258762,258763,258764,258765,258766] 2024/11/23-12:03:14.558373 139957383018048 (Original Log Time 2024/11/23-12:02:31.704292) EVENT_LOG_v1 {"time_micros": 1732334551704233, "job": 25666, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [258701], "files_L3": [258750, 258751, 258642, 258643, 258664, 258665, 258666, 258667, 258668, 258669, 258670, 258671, 258739], "score": 0.997871, "input_data_size": 488517593, "oldest_snapshot_seqno": 3964903723} 2024/11/23-12:03:14.558376 139957383018048 (Original Log Time 2024/11/23-12:03:14.557281) [compaction_job.cc:2022] [default] [JOB 25666] Compacted 1@2 + 13@3 files to L3 => 453256205 bytes 2024/11/23-12:03:14.558378 139957383018048 (Original Log Time 2024/11/23-12:03:14.558301) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 13 0 0 0] max score 0.02, MB/sec: 11.4 rd, 10.6 wr, level 3, files in(1, 13) out(13 +0 blob) MB in(27.1, 438.8 +0.0 blob) out(432.3 +0.0 blob), read-write-amplify(33.2) write-amplify(16.0) OK, records in: 11637706, records dropped: 32534 output_compression: Snappy 2024/11/23-12:03:14.558383 139957383018048 (Original Log Time 2024/11/23-12:03:14.558353) EVENT_LOG_v1 {"time_micros": 1732334594558324, "cf": "default", "job": 25666, "event": "compaction_finished", "compaction_time_micros": 42776350, "compaction_time_cpu_micros": 17398508, "output_level": 3, "num_output_files": 13, "total_output_size": 453256205, "num_input_records": 11637706, "num_output_records": 11605172, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 308449445, "file_range_sync_nanos": 0, "file_fsync_nanos": 9023, "file_prepare_write_nanos": 31799, "lsm_state": [0, 0, 0, 13, 0, 0, 0]} 2024/11/23-12:03:14.685351 139957383018048 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732334551,"executesMs":43019,"compactionJobId":25666,"attempt":0,"compactionInputRawBytes":1680315276,"compactionInputZipBytes":488517593,"compactionOutputRawBytes":0,"compactionOutputZipBytes":453256205,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-687479a05d63c3c2ec3c5061c7e0bfdc-520eaff6e93d1da5-00"}}, response = 2024/11/23-12:03:14.685459 139957383018048 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-25666/att-00: olev 3, work 43.020 s, result 7.708 ms, install 1.117 ms, input{raw 1.565 GiB zip 465.887 MiB}