2024/11/23-08:38:47.408308 140024844146240 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00: fork to child time = 0.009059 sec 2024/11/23-08:38:47.409013 140024844146240 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00 2024/11/23-08:38:47.415859 140024844146240 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00 2024/11/23-08:38:47.416969 140024844146240 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-08:38:47.417430 140024844146240 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-08:38:47.459789 140024844146240 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-22862/att-00/MANIFEST-206347 2024/11/23-08:38:47.487758 140024844146240 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-22862/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 252736, last_sequence is 3888291040, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 252554 2024/11/23-08:38:47.487769 140024844146240 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-08:38:47.487880 140024844146240 [version_set.cc:5721] Creating manifest 252736 2024/11/23-08:38:47.519596 140024844146240 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-08:38:47.540335 140024844146240 [compaction_job.cc:2437] [default] [JOB 22862] Compacting 1@2 + 11@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-08:38:47.540356 140024844146240 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 17170 Base level 2, inputs: [252693(27MB)], [252639(36MB) 252640(37MB) 252641(36MB) 252642(35MB) 252643(36MB) 252644(35MB) 252645(34MB) 252646(34MB) 252647(34MB) 252648(36MB) 252722(35MB)] 2024/11/23-08:38:47.546230 140024844146240 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-08:38:47.546269 140024844146240 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-08:38:47.546288 140024844146240 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-08:38:47.546294 140024844146240 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-08:38:47.546306 140024844146240 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-08:38:48.200933 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252737.sst) /dev/shm/ToplingTemp/Topling-462488-3i95Hl 2024/11/23-08:38:48.757839 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252737.sst): first pass time = 0.56's, 241.025'MB/sec 2024/11/23-08:38:50.121970 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252737.sst): old prealloc_size = 147639500, real_size = 34179232 2024/11/23-08:38:50.122536 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252737.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926482 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0322 GB avg-val = 116.87 avg-zval = 34.73 2024/11/23-08:38:50.123339 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252737.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:38:51.124156 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252737.sst): second pass time = 1.36's, 79.416'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.00's, 38.193'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3246.046'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.266'MB/sec write SST data time = 0.13's, 294.453'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33135.250'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.416'MB/sec zip pipeline throughput = 79.416'MB/sec entries = 926482 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.87 avg-zval = 34.73 usrkeys = 926482 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.87 avg-zval = 34.73 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926482 bytes = 3242704 } seq expand size = 6485374 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0322 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.4450 value = 3.3654 dict = 1.00 all = 3.5066 } Zip/UnZip{ index = 0.0957 value = 0.2971 dict = 1.00 all = 0.2852 } ---------------------------- total value len = 0.114762 GB avg = 0.124 KB (by entry num) total key len = 0.025941 GB avg = 0.028 KB total ukey len = 0.018530 GB avg = 0.020 KB total ukey num = 0.000926482 Billion total entry num = 0.000926482 Billion write speed all = 0.001643959 MB/sec (with seq num) write speed all = 0.001557360 MB/sec (without seq num) 2024/11/23-08:38:51.158697 140024844146240 [compaction_job.cc:1948] [default] [JOB 22862] Generated table #252737: 926482 keys, 38276288 bytes, temperature: kWarm 2024/11/23-08:38:51.158801 140024844146240 EVENT_LOG_v1 {"time_micros": 1732322331158740, "cf_name": "default", "job": 22862, "event": "table_file_creation", "file_number": 252737, "file_size": 38276288, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3711387651, "largest_seqno": 3888143051, "table_properties": {"data_size": 32173392, "index_size": 1774016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25941496, "raw_average_key_size": 28, "raw_value_size": 108276368, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926482, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732322328, "oldest_key_time": 0, "file_creation_time": 1732322328, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 252737, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:38:51.197178 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252738.sst) /dev/shm/ToplingTemp/Topling-462488-iw1Kil 2024/11/23-08:38:52.124740 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252738.sst): first pass time = 0.93's, 144.706'MB/sec 2024/11/23-08:38:53.487199 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252738.sst): old prealloc_size = 147639500, real_size = 36055040 2024/11/23-08:38:53.487800 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252738.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925175 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0341 GB avg-val = 117.07 avg-zval = 36.81 2024/11/23-08:38:53.488567 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252738.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:38:54.441959 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252738.sst): second pass time = 1.36's, 79.555'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 41.930'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3071.052'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.280'MB/sec write SST data time = 0.13's, 310.455'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35500.493'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.555'MB/sec zip pipeline throughput = 79.555'MB/sec entries = 925175 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.07 avg-zval = 36.81 usrkeys = 925175 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.07 avg-zval = 36.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925175 bytes = 3122480 } seq expand size = 6476225 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0341 GB dict = 1.08 MB all = 0.0400 GB } UnZip/Zip{ index = 10.4412 value = 3.1809 dict = 1.00 all = 3.3527 } Zip/UnZip{ index = 0.0958 value = 0.3144 dict = 1.00 all = 0.2983 } ---------------------------- total value len = 0.229551 GB avg = 0.124 KB (by entry num) total key len = 0.051846 GB avg = 0.028 KB total ukey len = 0.037033 GB avg = 0.020 KB total ukey num = 0.001851657 Billion total entry num = 0.001851657 Billion write speed all = 0.003287683 MB/sec (with seq num) write speed all = 0.003114614 MB/sec (without seq num) 2024/11/23-08:38:54.479626 140024844146240 [compaction_job.cc:1948] [default] [JOB 22862] Generated table #252738: 925175 keys, 40032580 bytes, temperature: kWarm 2024/11/23-08:38:54.479696 140024844146240 EVENT_LOG_v1 {"time_micros": 1732322334479664, "cf_name": "default", "job": 22862, "event": "table_file_creation", "file_number": 252738, "file_size": 40032580, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3755579279, "largest_seqno": 3888143467, "table_properties": {"data_size": 34051376, "index_size": 1772160, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25904900, "raw_average_key_size": 28, "raw_value_size": 108312878, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925175, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732322331, "oldest_key_time": 0, "file_creation_time": 1732322331, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 252738, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:38:54.516817 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252739.sst) /dev/shm/ToplingTemp/Topling-462488-6F2ORn 2024/11/23-08:38:55.399324 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252739.sst): first pass time = 0.88's, 152.094'MB/sec 2024/11/23-08:38:56.795266 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252739.sst): old prealloc_size = 147639500, real_size = 34742432 2024/11/23-08:38:56.795838 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252739.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927591 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0327 GB avg-val = 116.70 avg-zval = 35.30 2024/11/23-08:38:56.796607 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252739.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:38:57.798335 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252739.sst): second pass time = 1.40's, 77.583'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.00's, 38.841'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3231.393'MB/sec (index lex order gen) rebuild zvType time = 0.85's, 0.272'MB/sec write SST data time = 0.15's, 262.056'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 27621.553'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.583'MB/sec zip pipeline throughput = 77.583'MB/sec entries = 927591 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.70 avg-zval = 35.30 usrkeys = 927591 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.70 avg-zval = 35.30 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927591 bytes = 3362528 } seq expand size = 6493137 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0327 GB dict = 1.08 MB all = 0.0390 GB } UnZip/Zip{ index = 10.5137 value = 3.3056 dict = 1.00 all = 3.4451 } Zip/UnZip{ index = 0.0951 value = 0.3025 dict = 1.00 all = 0.2903 } ---------------------------- total value len = 0.344289 GB avg = 0.124 KB (by entry num) total key len = 0.077819 GB avg = 0.028 KB total ukey len = 0.055585 GB avg = 0.020 KB total ukey num = 0.002779248 Billion total entry num = 0.002779248 Billion write speed all = 0.004931476 MB/sec (with seq num) write speed all = 0.004671717 MB/sec (without seq num) 2024/11/23-08:38:57.833231 140024844146240 [compaction_job.cc:1948] [default] [JOB 22862] Generated table #252739: 927591 keys, 38959502 bytes, temperature: kWarm 2024/11/23-08:38:57.833292 140024844146240 EVENT_LOG_v1 {"time_micros": 1732322337833263, "cf_name": "default", "job": 22862, "event": "table_file_creation", "file_number": 252739, "file_size": 38959502, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038136, "largest_seqno": 3888143087, "table_properties": {"data_size": 32745808, "index_size": 1764544, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25972548, "raw_average_key_size": 28, "raw_value_size": 108245294, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927591, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732322334, "oldest_key_time": 0, "file_creation_time": 1732322334, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 252739, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:38:57.867581 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252740.sst) /dev/shm/ToplingTemp/Topling-462488-vZZFil 2024/11/23-08:38:58.677350 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252740.sst): first pass time = 0.81's, 165.759'MB/sec 2024/11/23-08:38:59.975078 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252740.sst): old prealloc_size = 147639500, real_size = 33619600 2024/11/23-08:38:59.975617 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252740.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925088 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 117.09 avg-zval = 34.19 2024/11/23-08:38:59.976361 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252740.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:39:00.927885 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252740.sst): second pass time = 1.30's, 83.506'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.581'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3396.616'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.281'MB/sec write SST data time = 0.13's, 290.679'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35676.040'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.506'MB/sec zip pipeline throughput = 83.506'MB/sec entries = 925088 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.09 avg-zval = 34.19 usrkeys = 925088 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.09 avg-zval = 34.19 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925088 bytes = 3237824 } seq expand size = 6475616 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5105 value = 3.4247 dict = 1.00 all = 3.5589 } Zip/UnZip{ index = 0.0951 value = 0.2920 dict = 1.00 all = 0.2810 } ---------------------------- total value len = 0.459080 GB avg = 0.124 KB (by entry num) total key len = 0.103721 GB avg = 0.028 KB total ukey len = 0.074087 GB avg = 0.020 KB total ukey num = 0.003704336 Billion total entry num = 0.003704336 Billion write speed all = 0.006574952 MB/sec (with seq num) write speed all = 0.006228744 MB/sec (without seq num) 2024/11/23-08:39:00.966303 140024844146240 [compaction_job.cc:1948] [default] [JOB 22862] Generated table #252740: 925088 keys, 37713000 bytes, temperature: kWarm 2024/11/23-08:39:00.966364 140024844146240 EVENT_LOG_v1 {"time_micros": 1732322340966333, "cf_name": "default", "job": 22862, "event": "table_file_creation", "file_number": 252740, "file_size": 37713000, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3744901358, "largest_seqno": 3888143569, "table_properties": {"data_size": 31627840, "index_size": 1760320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25902464, "raw_average_key_size": 28, "raw_value_size": 108315372, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925088, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732322337, "oldest_key_time": 0, "file_creation_time": 1732322337, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 252740, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:39:01.003789 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252741.sst) /dev/shm/ToplingTemp/Topling-462488-xMnh8m 2024/11/23-08:39:01.985469 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252741.sst): first pass time = 0.98's, 136.728'MB/sec 2024/11/23-08:39:04.052971 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252741.sst): old prealloc_size = 147639500, real_size = 34919248 2024/11/23-08:39:04.053584 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252741.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933035 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0329 GB avg-val = 115.85 avg-zval = 35.25 2024/11/23-08:39:04.054126 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252741.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:39:05.270033 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252741.sst): second pass time = 2.07's, 52.307'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.22's, 30.948'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3046.292'MB/sec (index lex order gen) rebuild zvType time = 1.00's, 0.233'MB/sec write SST data time = 0.21's, 175.367'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 30380.324'MB/sec zip my value time = 2.07's, unzip length = 0.108'GB zip my value throughput = 52.307'MB/sec zip pipeline throughput = 92.264'MB/sec entries = 933035 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.85 avg-zval = 35.25 usrkeys = 933035 avg-key = 20.00 avg-zkey = 1.93 avg-val = 119.35 avg-zval = 35.25 TagRS{ kinds = RS_Key0_TagN bytes = 145880 } TagArray{ size = 465781 bytes = 1746688 } seq expand size = 3260467 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0329 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.3770 value = 3.2867 dict = 1.00 all = 3.5634 } Zip/UnZip{ index = 0.0964 value = 0.3043 dict = 1.00 all = 0.2806 } ---------------------------- total value len = 0.570434 GB avg = 0.123 KB (by entry num) total key len = 0.129846 GB avg = 0.028 KB total ukey len = 0.092747 GB avg = 0.020 KB total ukey num = 0.004637371 Billion total entry num = 0.004637371 Billion write speed all = 0.008180633 MB/sec (with seq num) write speed all = 0.007747245 MB/sec (without seq num) 2024/11/23-08:39:05.303840 140024844146240 [compaction_job.cc:1948] [default] [JOB 22862] Generated table #252741: 933035 keys, 37665853 bytes, temperature: kWarm 2024/11/23-08:39:05.303905 140024844146240 EVENT_LOG_v1 {"time_micros": 1732322345303869, "cf_name": "default", "job": 22862, "event": "table_file_creation", "file_number": 252741, "file_size": 37665853, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3888141718, "table_properties": {"data_size": 32887552, "index_size": 1798272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26124980, "raw_average_key_size": 28, "raw_value_size": 108092874, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933035, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732322340, "oldest_key_time": 0, "file_creation_time": 1732322340, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 252741, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:39:05.343812 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252742.sst) /dev/shm/ToplingTemp/Topling-462488-hnrdTm 2024/11/23-08:39:06.261826 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252742.sst): first pass time = 0.92's, 146.211'MB/sec 2024/11/23-08:39:08.506984 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252742.sst): old prealloc_size = 147639500, real_size = 36432672 2024/11/23-08:39:08.507589 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252742.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927099 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0344 GB avg-val = 116.77 avg-zval = 37.14 2024/11/23-08:39:08.507647 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252742.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:39:09.278381 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252742.sst): second pass time = 2.24's, 48.243'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.77's, 48.342'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3037.254'MB/sec (index lex order gen) rebuild zvType time = 0.62's, 0.375'MB/sec write SST data time = 0.15's, 244.567'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31943.934'MB/sec zip my value time = 2.24's, unzip length = 0.108'GB zip my value throughput = 48.243'MB/sec zip pipeline throughput = 61.239'MB/sec entries = 927099 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.77 avg-zval = 37.14 usrkeys = 927099 avg-key = 20.00 avg-zkey = 1.90 avg-val = 116.77 avg-zval = 37.14 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5127 value = 3.1437 dict = 1.00 all = 3.5992 } Zip/UnZip{ index = 0.0951 value = 0.3181 dict = 1.00 all = 0.2778 } ---------------------------- total value len = 0.678693 GB avg = 0.122 KB (by entry num) total key len = 0.155805 GB avg = 0.028 KB total ukey len = 0.111289 GB avg = 0.020 KB total ukey num = 0.005564470 Billion total entry num = 0.005564470 Billion write speed all = 0.009748101 MB/sec (with seq num) write speed all = 0.009228095 MB/sec (without seq num) 2024/11/23-08:39:09.316259 140024844146240 [compaction_job.cc:1948] [default] [JOB 22862] Generated table #252742: 927099 keys, 37290618 bytes, temperature: kWarm 2024/11/23-08:39:09.316322 140024844146240 EVENT_LOG_v1 {"time_micros": 1732322349316291, "cf_name": "default", "job": 22862, "event": "table_file_creation", "file_number": 252742, "file_size": 37290618, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34436944, "index_size": 1763776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25958772, "raw_average_key_size": 28, "raw_value_size": 108258970, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927099, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732322345, "oldest_key_time": 0, "file_creation_time": 1732322345, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 252742, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:39:09.353245 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252743.sst) /dev/shm/ToplingTemp/Topling-462488-mRZ15l 2024/11/23-08:39:10.535513 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252743.sst): first pass time = 1.18's, 113.529'MB/sec 2024/11/23-08:39:12.465043 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252743.sst): old prealloc_size = 147639500, real_size = 33633616 2024/11/23-08:39:12.465647 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252743.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924986 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 117.10 avg-zval = 34.20 2024/11/23-08:39:12.466324 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252743.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:39:13.364596 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252743.sst): second pass time = 1.93's, 56.157'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 41.381'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3041.748'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.309'MB/sec write SST data time = 0.15's, 248.488'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 28406.727'MB/sec zip my value time = 1.93's, unzip length = 0.108'GB zip my value throughput = 56.157'MB/sec zip pipeline throughput = 108.148'MB/sec entries = 924986 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.10 avg-zval = 34.20 usrkeys = 924986 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.32 avg-zval = 34.20 TagRS{ kinds = RS_Key0_TagN bytes = 144640 } TagArray{ size = 689848 bytes = 2586944 } seq expand size = 4828936 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.4425 value = 3.4245 dict = 1.00 all = 3.6056 } Zip/UnZip{ index = 0.0958 value = 0.2920 dict = 1.00 all = 0.2773 } ---------------------------- total value len = 0.791840 GB avg = 0.122 KB (by entry num) total key len = 0.181705 GB avg = 0.028 KB total ukey len = 0.129789 GB avg = 0.020 KB total ukey num = 0.006489456 Billion total entry num = 0.006489456 Billion write speed all = 0.011371819 MB/sec (with seq num) write speed all = 0.010765400 MB/sec (without seq num) 2024/11/23-08:39:13.400480 140024844146240 [compaction_job.cc:1948] [default] [JOB 22862] Generated table #252743: 924986 keys, 37224458 bytes, temperature: kWarm 2024/11/23-08:39:13.400543 140024844146240 EVENT_LOG_v1 {"time_micros": 1732322353400512, "cf_name": "default", "job": 22862, "event": "table_file_creation", "file_number": 252743, "file_size": 37224458, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3888143246, "table_properties": {"data_size": 31630592, "index_size": 1771584, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25899608, "raw_average_key_size": 28, "raw_value_size": 108318196, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924986, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732322349, "oldest_key_time": 0, "file_creation_time": 1732322349, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 252743, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:39:13.438845 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252744.sst) /dev/shm/ToplingTemp/Topling-462488-p0bQap 2024/11/23-08:39:14.412013 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252744.sst): first pass time = 0.97's, 137.924'MB/sec 2024/11/23-08:39:16.426030 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252744.sst): old prealloc_size = 147639500, real_size = 33051184 2024/11/23-08:39:16.426608 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252744.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925449 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 117.03 avg-zval = 33.55 2024/11/23-08:39:16.427391 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252744.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:39:17.516037 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252744.sst): second pass time = 2.01's, 53.800'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.09's, 34.185'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3168.400'MB/sec (index lex order gen) rebuild zvType time = 0.86's, 0.270'MB/sec write SST data time = 0.23's, 159.239'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32431.556'MB/sec zip my value time = 2.01's, unzip length = 0.108'GB zip my value throughput = 53.800'MB/sec zip pipeline throughput = 146.129'MB/sec entries = 925449 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.03 avg-zval = 33.55 usrkeys = 925449 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.03 avg-zval = 33.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925449 bytes = 3354768 } seq expand size = 6478143 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4432 value = 3.4884 dict = 1.00 all = 3.6021 } Zip/UnZip{ index = 0.0958 value = 0.2867 dict = 1.00 all = 0.2776 } ---------------------------- total value len = 0.906623 GB avg = 0.122 KB (by entry num) total key len = 0.207617 GB avg = 0.028 KB total ukey len = 0.148298 GB avg = 0.020 KB total ukey num = 0.007414905 Billion total entry num = 0.007414905 Billion write speed all = 0.013014635 MB/sec (with seq num) write speed all = 0.012321770 MB/sec (without seq num) 2024/11/23-08:39:17.553770 140024844146240 [compaction_job.cc:1948] [default] [JOB 22862] Generated table #252744: 925449 keys, 37260792 bytes, temperature: kWarm 2024/11/23-08:39:17.553861 140024844146240 EVENT_LOG_v1 {"time_micros": 1732322357553817, "cf_name": "default", "job": 22862, "event": "table_file_creation", "file_number": 252744, "file_size": 37260792, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3509327862, "largest_seqno": 3888142070, "table_properties": {"data_size": 31047264, "index_size": 1772352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25912572, "raw_average_key_size": 28, "raw_value_size": 108305262, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925449, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732322353, "oldest_key_time": 0, "file_creation_time": 1732322353, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 252744, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:39:17.609625 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252745.sst) /dev/shm/ToplingTemp/Topling-462488-CcjXsm 2024/11/23-08:39:18.468938 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252745.sst): first pass time = 0.86's, 156.201'MB/sec 2024/11/23-08:39:19.904063 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252745.sst): old prealloc_size = 147639500, real_size = 32744000 2024/11/23-08:39:19.904650 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252745.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931729 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0307 GB avg-val = 116.05 avg-zval = 32.98 2024/11/23-08:39:19.905424 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252745.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:39:20.865337 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252745.sst): second pass time = 1.43's, 75.381'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 38.342'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3161.289'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.280'MB/sec write SST data time = 0.13's, 287.114'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 29058.456'MB/sec zip my value time = 1.43's, unzip length = 0.108'GB zip my value throughput = 75.381'MB/sec zip pipeline throughput = 92.991'MB/sec entries = 931729 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.05 avg-zval = 32.98 usrkeys = 931729 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.05 avg-zval = 32.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931729 bytes = 3261072 } seq expand size = 6522103 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4435 value = 3.5191 dict = 1.00 all = 3.6417 } Zip/UnZip{ index = 0.0958 value = 0.2842 dict = 1.00 all = 0.2746 } ---------------------------- total value len = 1.021275 GB avg = 0.122 KB (by entry num) total key len = 0.233706 GB avg = 0.028 KB total ukey len = 0.166933 GB avg = 0.020 KB total ukey num = 0.008346634 Billion total entry num = 0.008346634 Billion write speed all = 0.014657943 MB/sec (with seq num) write speed all = 0.013878046 MB/sec (without seq num) 2024/11/23-08:39:20.901121 140024844146240 [compaction_job.cc:1948] [default] [JOB 22862] Generated table #252745: 931729 keys, 36856150 bytes, temperature: kWarm 2024/11/23-08:39:20.901182 140024844146240 EVENT_LOG_v1 {"time_micros": 1732322360901151, "cf_name": "default", "job": 22862, "event": "table_file_creation", "file_number": 252745, "file_size": 36856150, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3631595348, "largest_seqno": 3888143274, "table_properties": {"data_size": 30726576, "index_size": 1784320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26088412, "raw_average_key_size": 28, "raw_value_size": 108129410, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931729, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732322357, "oldest_key_time": 0, "file_creation_time": 1732322357, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 252745, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:39:20.940621 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252746.sst) /dev/shm/ToplingTemp/Topling-462488-QmIX0k 2024/11/23-08:39:21.869384 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252746.sst): first pass time = 0.93's, 144.519'MB/sec 2024/11/23-08:39:23.281135 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252746.sst): old prealloc_size = 147639500, real_size = 34453488 2024/11/23-08:39:23.281719 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252746.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929666 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0324 GB avg-val = 116.37 avg-zval = 34.89 2024/11/23-08:39:23.282533 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252746.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:39:24.196919 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252746.sst): second pass time = 1.41's, 76.674'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 42.233'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3163.286'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.296'MB/sec write SST data time = 0.13's, 296.596'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35993.329'MB/sec zip my value time = 1.41's, unzip length = 0.108'GB zip my value throughput = 76.674'MB/sec zip pipeline throughput = 76.674'MB/sec entries = 929666 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.37 avg-zval = 34.89 usrkeys = 929666 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.37 avg-zval = 34.89 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929666 bytes = 3370048 } seq expand size = 6507662 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0324 GB dict = 1.08 MB all = 0.0387 GB } UnZip/Zip{ index = 10.4440 value = 3.3349 dict = 1.00 all = 3.4703 } Zip/UnZip{ index = 0.0957 value = 0.2999 dict = 1.00 all = 0.2882 } ---------------------------- total value len = 1.135969 GB avg = 0.122 KB (by entry num) total key len = 0.259736 GB avg = 0.028 KB total ukey len = 0.185526 GB avg = 0.020 KB total ukey num = 0.009276300 Billion total entry num = 0.009276300 Billion write speed all = 0.016300957 MB/sec (with seq num) write speed all = 0.015434227 MB/sec (without seq num) 2024/11/23-08:39:24.235483 140024844146240 [compaction_job.cc:1948] [default] [JOB 22862] Generated table #252746: 929666 keys, 38676096 bytes, temperature: kWarm 2024/11/23-08:39:24.235548 140024844146240 EVENT_LOG_v1 {"time_micros": 1732322364235516, "cf_name": "default", "job": 22862, "event": "table_file_creation", "file_number": 252746, "file_size": 38676096, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 3888143288, "table_properties": {"data_size": 32440608, "index_size": 1780288, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26030648, "raw_average_key_size": 28, "raw_value_size": 108187200, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929666, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732322360, "oldest_key_time": 0, "file_creation_time": 1732322360, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 252746, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:39:24.271754 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252747.sst) /dev/shm/ToplingTemp/Topling-462488-ONSEko 2024/11/23-08:39:25.113569 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252747.sst): first pass time = 0.84's, 159.447'MB/sec 2024/11/23-08:39:26.523325 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252747.sst): old prealloc_size = 147639500, real_size = 32421600 2024/11/23-08:39:26.523890 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252747.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929564 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0304 GB avg-val = 116.39 avg-zval = 32.73 2024/11/23-08:39:26.524664 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252747.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:39:27.441910 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252747.sst): second pass time = 1.41's, 76.783'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 39.766'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3248.393'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.294'MB/sec write SST data time = 0.13's, 286.288'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 23318.932'MB/sec zip my value time = 1.41's, unzip length = 0.108'GB zip my value throughput = 76.783'MB/sec zip pipeline throughput = 89.917'MB/sec entries = 929564 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.39 avg-zval = 32.73 usrkeys = 929564 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.39 avg-zval = 32.73 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929564 bytes = 3253488 } seq expand size = 6506948 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5139 value = 3.5565 dict = 1.00 all = 3.6744 } Zip/UnZip{ index = 0.0951 value = 0.2812 dict = 1.00 all = 0.2722 } ---------------------------- total value len = 1.250666 GB avg = 0.123 KB (by entry num) total key len = 0.285764 GB avg = 0.028 KB total ukey len = 0.204117 GB avg = 0.020 KB total ukey num = 0.010205864 Billion total entry num = 0.010205864 Billion write speed all = 0.017943853 MB/sec (with seq num) write speed all = 0.016990305 MB/sec (without seq num) 2024/11/23-08:39:27.479839 140024844146240 [compaction_job.cc:1948] [default] [JOB 22862] Generated table #252747: 929564 keys, 36527944 bytes, temperature: kWarm 2024/11/23-08:39:27.479899 140024844146240 EVENT_LOG_v1 {"time_micros": 1732322367479869, "cf_name": "default", "job": 22862, "event": "table_file_creation", "file_number": 252747, "file_size": 36527944, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3621075956, "largest_seqno": 3888143305, "table_properties": {"data_size": 30420752, "index_size": 1768256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26027792, "raw_average_key_size": 28, "raw_value_size": 108190016, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929564, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732322364, "oldest_key_time": 0, "file_creation_time": 1732322364, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 252747, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:39:27.516749 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252748.sst) /dev/shm/ToplingTemp/Topling-462488-DNizRk 2024/11/23-08:39:27.633732 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252748.sst): first pass time = 0.12's, 233.916'MB/sec 2024/11/23-08:39:27.895333 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252748.sst): old prealloc_size = 147639500, real_size = 7891680 2024/11/23-08:39:27.895489 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252748.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 187346 prefix = 4 raw-key = 0.0030 GB zip-key = 0.0004 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0221 GB zip-val = 0.0075 GB avg-val = 118.00 avg-zval = 39.95 2024/11/23-08:39:27.895663 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252748.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0004 GB, waited 0.000 sec, Key+Value = 0.027 GB 2024/11/23-08:39:28.074791 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00/252748.sst): second pass time = 0.26's, 84.674'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 48.412'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2734.532'MB/sec (index lex order gen) rebuild zvType time = 0.15's, 0.306'MB/sec write SST data time = 0.03's, 329.182'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.232'MB, throughput = 5673.160'MB/sec zip my value time = 0.26's, unzip length = 0.022'GB zip my value throughput = 84.674'MB/sec zip pipeline throughput = 84.674'MB/sec entries = 187346 avg-key = 28.00 avg-zkey = 1.92 avg-val = 118.00 avg-zval = 39.95 usrkeys = 187346 avg-key = 20.00 avg-zkey = 1.92 avg-val = 125.00 avg-zval = 39.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 187346 bytes = 608896 } seq expand size = 1311422 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0037 GB value = 0.0221 GB dict = 0.23 MB all = 0.0274 GB } __ZipSize{ index = 0.0004 GB value = 0.0075 GB dict = 0.23 MB all = 0.0087 GB } UnZip/Zip{ index = 10.4063 value = 2.9536 dict = 1.00 all = 3.1487 } Zip/UnZip{ index = 0.0961 value = 0.3386 dict = 1.00 all = 0.3176 } ---------------------------- total value len = 1.274085 GB avg = 0.123 KB (by entry num) total key len = 0.291010 GB avg = 0.028 KB total ukey len = 0.207864 GB avg = 0.020 KB total ukey num = 0.010393210 Billion total entry num = 0.010393210 Billion write speed all = 0.018278481 MB/sec (with seq num) write speed all = 0.017307437 MB/sec (without seq num) 2024/11/23-08:39:28.089001 140024844146240 [compaction_job.cc:1948] [default] [JOB 22862] Generated table #252748: 187346 keys, 8686843 bytes, temperature: kWarm 2024/11/23-08:39:28.089059 140024844146240 EVENT_LOG_v1 {"time_micros": 1732322368089029, "cf_name": "default", "job": 22862, "event": "table_file_creation", "file_number": 252748, "file_size": 8686843, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3803386309, "largest_seqno": 3841080199, "table_properties": {"data_size": 7484624, "index_size": 360064, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 5245688, "raw_average_key_size": 28, "raw_value_size": 22106828, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 187346, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732322367, "oldest_key_time": 0, "file_creation_time": 1732322367, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 252748, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:39:28.100509 140024844146240 [compaction_job.cc:703] job-22862: subcompact[0], size: 0.425170 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-22862/att-00 [252737,252738,252739,252740,252741,252742,252743,252744,252745,252746,252747,252748] 2024/11/23-08:39:28.179055 140024844146240 (Original Log Time 2024/11/23-08:38:47.540416) EVENT_LOG_v1 {"time_micros": 1732322327540373, "job": 22862, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [252693], "files_L3": [252639, 252640, 252641, 252642, 252643, 252644, 252645, 252646, 252647, 252648, 252722], "score": 0.99722, "input_data_size": 441670717, "oldest_snapshot_seqno": 3888291010} 2024/11/23-08:39:28.179058 140024844146240 (Original Log Time 2024/11/23-08:39:28.178352) [compaction_job.cc:2022] [default] [JOB 22862] Compacted 1@2 + 11@3 files to L3 => 425170124 bytes 2024/11/23-08:39:28.179060 140024844146240 (Original Log Time 2024/11/23-08:39:28.178987) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 12 0 0 0] max score 0.02, MB/sec: 10.9 rd, 10.5 wr, level 3, files in(1, 11) out(12 +0 blob) MB in(27.1, 394.1 +0.0 blob) out(405.5 +0.0 blob), read-write-amplify(30.5) write-amplify(15.0) OK, records in: 10436479, records dropped: 43269 output_compression: Snappy 2024/11/23-08:39:28.179066 140024844146240 (Original Log Time 2024/11/23-08:39:28.179034) EVENT_LOG_v1 {"time_micros": 1732322368179011, "cf": "default", "job": 22862, "event": "compaction_finished", "compaction_time_micros": 40560102, "compaction_time_cpu_micros": 17514661, "output_level": 3, "num_output_files": 12, "total_output_size": 425170124, "num_input_records": 10436479, "num_output_records": 10393210, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 289737454, "file_range_sync_nanos": 0, "file_fsync_nanos": 10500, "file_prepare_write_nanos": 30101, "lsm_state": [0, 0, 0, 12, 0, 0, 0]} 2024/11/23-08:39:28.301386 140024844146240 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732322327,"executesMs":40763,"compactionJobId":22862,"attempt":0,"compactionInputRawBytes":1509892746,"compactionInputZipBytes":441670717,"compactionOutputRawBytes":0,"compactionOutputZipBytes":425170124,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-0fd09ebebd338155098b5a6c5190dd6f-fd145b9fe2af5710-00"}}, response = 2024/11/23-08:39:28.301463 140024844146240 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-22862/att-00: olev 3, work 40.763 s, result 6.346 ms, install 0.736 ms, input{raw 1.406 GiB zip 421.210 MiB}