2024/11/23-20:17:11.529498 139957164807744 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00: fork to child time = 0.007777 sec 2024/11/23-20:17:11.530289 139957164807744 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00 2024/11/23-20:17:11.537681 139957164807744 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00 2024/11/23-20:17:11.538588 139957164807744 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-20:17:11.538847 139957164807744 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-20:17:11.588515 139957164807744 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-35016/att-00/MANIFEST-206347 2024/11/23-20:17:11.608490 139957164807744 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-35016/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 278961, last_sequence is 4239518775, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 278816 2024/11/23-20:17:11.608497 139957164807744 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-20:17:11.608665 139957164807744 [version_set.cc:5721] Creating manifest 278961 2024/11/23-20:17:11.650211 139957164807744 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-20:17:11.673287 139957164807744 [compaction_job.cc:2437] [default] [JOB 35016] Compacting 1@2 + 18@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-20:17:11.673320 139957164807744 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 26209 Base level 2, inputs: [278955(27MB)], [278893(36MB) 278894(36MB) 278895(35MB) 278896(36MB) 278897(38MB) 278898(37MB) 278899(38MB) 278900(35MB) 278901(34MB) 278902(33MB) 278903(35MB) 278904(36MB) 278905(36MB) 278906(35MB) 278907(35MB) 278908(36MB) 278909(37MB) 278918(36MB)] 2024/11/23-20:17:11.680696 139957164807744 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-20:17:11.680742 139957164807744 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-20:17:11.680772 139957164807744 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-20:17:11.680783 139957164807744 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-20:17:11.680799 139957164807744 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-20:17:12.398885 139957164807744 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278962.sst) /dev/shm/ToplingTemp/Topling-177538-NiF92M 2024/11/23-20:17:13.032870 139957164807744 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278962.sst): first pass time = 0.63's, 211.723'MB/sec 2024/11/23-20:17:15.023728 139957164807744 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278962.sst): old prealloc_size = 147639500, real_size = 34388224 2024/11/23-20:17:15.024311 139957164807744 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278962.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927269 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0324 GB avg-val = 116.75 avg-zval = 34.93 2024/11/23-20:17:15.025145 139957164807744 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278962.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:17:16.751302 139957164807744 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278962.sst): second pass time = 1.99's, 54.398'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.73's, 22.212'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3150.026'MB/sec (index lex order gen) rebuild zvType time = 1.36's, 0.170'MB/sec write SST data time = 0.36's, 105.753'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 36455.616'MB/sec zip my value time = 1.99's, unzip length = 0.108'GB zip my value throughput = 54.398'MB/sec zip pipeline throughput = 121.153'MB/sec entries = 927269 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.75 avg-zval = 34.93 usrkeys = 927269 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.75 avg-zval = 34.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927269 bytes = 3129552 } seq expand size = 6490883 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0324 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.5150 value = 3.3420 dict = 1.00 all = 3.4978 } Zip/UnZip{ index = 0.0951 value = 0.2992 dict = 1.00 all = 0.2859 } ---------------------------- total value len = 8.963365 GB avg = 0.168 KB (by entry num) total key len = 1.466785 GB avg = 0.028 KB total ukey len = 1.016423 GB avg = 0.015 KB total ukey num = 0.066477704 Billion total entry num = 0.053285433 Billion write speed all = 0.254862983 MB/sec (with seq num) write speed all = 0.244446654 MB/sec (without seq num) 2024/11/23-20:17:16.804652 139957164807744 [compaction_job.cc:1948] [default] [JOB 35016] Generated table #278962: 927269 keys, 38372300 bytes, temperature: kWarm 2024/11/23-20:17:16.804778 139957164807744 EVENT_LOG_v1 {"time_micros": 1732364236804700, "cf_name": "default", "job": 35016, "event": "table_file_creation", "file_number": 278962, "file_size": 38372300, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4136224572, "largest_seqno": 4239498388, "table_properties": {"data_size": 32392496, "index_size": 1763712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25963532, "raw_average_key_size": 28, "raw_value_size": 108254210, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927269, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364232, "oldest_key_time": 0, "file_creation_time": 1732364232, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278962, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:17:16.877938 139957164807744 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278963.sst) /dev/shm/ToplingTemp/Topling-177538-mmqLgM 2024/11/23-20:17:18.075341 139957164807744 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278963.sst): first pass time = 1.20's, 112.100'MB/sec 2024/11/23-20:17:19.962662 139957164807744 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278963.sst): old prealloc_size = 147639500, real_size = 33575280 2024/11/23-20:17:19.963230 139957164807744 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278963.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927384 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 116.73 avg-zval = 34.05 2024/11/23-20:17:19.964050 139957164807744 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278963.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:17:21.298929 139957164807744 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278963.sst): second pass time = 1.89's, 57.379'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.34's, 28.109'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3139.423'MB/sec (index lex order gen) rebuild zvType time = 1.09's, 0.214'MB/sec write SST data time = 0.25's, 150.326'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36256.486'MB/sec zip my value time = 1.89's, unzip length = 0.108'GB zip my value throughput = 57.379'MB/sec zip pipeline throughput = 66.523'MB/sec entries = 927384 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.73 avg-zval = 34.05 usrkeys = 927384 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.73 avg-zval = 34.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927384 bytes = 3129936 } seq expand size = 6491688 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5132 value = 3.4279 dict = 1.00 all = 3.5735 } Zip/UnZip{ index = 0.0951 value = 0.2917 dict = 1.00 all = 0.2798 } ---------------------------- total value len = 9.078108 GB avg = 0.167 KB (by entry num) total key len = 1.492751 GB avg = 0.028 KB total ukey len = 1.034971 GB avg = 0.015 KB total ukey num = 0.067405088 Billion total entry num = 0.054212817 Billion write speed all = 0.258272551 MB/sec (with seq num) write speed all = 0.247676112 MB/sec (without seq num) 2024/11/23-20:17:21.349248 139957164807744 [compaction_job.cc:1948] [default] [JOB 35016] Generated table #278963: 927384 keys, 37559510 bytes, temperature: kWarm 2024/11/23-20:17:21.349340 139957164807744 EVENT_LOG_v1 {"time_micros": 1732364241349290, "cf_name": "default", "job": 35016, "event": "table_file_creation", "file_number": 278963, "file_size": 37559510, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4168606255, "largest_seqno": 4239498449, "table_properties": {"data_size": 31579040, "index_size": 1764224, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25966752, "raw_average_key_size": 28, "raw_value_size": 108251056, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927384, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364236, "oldest_key_time": 0, "file_creation_time": 1732364236, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278963, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:17:21.426918 139957164807744 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278964.sst) /dev/shm/ToplingTemp/Topling-177538-wjES9K 2024/11/23-20:17:22.760764 139957164807744 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278964.sst): first pass time = 1.33's, 100.629'MB/sec 2024/11/23-20:17:25.010200 139957164807744 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278964.sst): old prealloc_size = 147639500, real_size = 33689888 2024/11/23-20:17:25.010898 139957164807744 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278964.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923155 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0317 GB avg-val = 117.39 avg-zval = 34.34 2024/11/23-20:17:25.011741 139957164807744 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278964.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:17:26.647177 139957164807744 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278964.sst): second pass time = 2.25's, 48.196'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.64's, 23.008'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2634.728'MB/sec (index lex order gen) rebuild zvType time = 1.37's, 0.168'MB/sec write SST data time = 0.26's, 143.769'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 36621.690'MB/sec zip my value time = 2.25's, unzip length = 0.108'GB zip my value throughput = 48.196'MB/sec zip pipeline throughput = 131.673'MB/sec entries = 923155 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.39 avg-zval = 34.34 usrkeys = 923155 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.39 avg-zval = 34.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923155 bytes = 3115664 } seq expand size = 6462085 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5110 value = 3.4183 dict = 1.00 all = 3.5636 } Zip/UnZip{ index = 0.0951 value = 0.2925 dict = 1.00 all = 0.2806 } ---------------------------- total value len = 9.192939 GB avg = 0.167 KB (by entry num) total key len = 1.518600 GB avg = 0.028 KB total ukey len = 1.053434 GB avg = 0.015 KB total ukey num = 0.068328243 Billion total entry num = 0.055135972 Billion write speed all = 0.261675519 MB/sec (with seq num) write speed all = 0.250900048 MB/sec (without seq num) 2024/11/23-20:17:26.706231 139957164807744 [compaction_job.cc:1948] [default] [JOB 35016] Generated table #278964: 923155 keys, 37663500 bytes, temperature: kWarm 2024/11/23-20:17:26.706329 139957164807744 EVENT_LOG_v1 {"time_micros": 1732364246706280, "cf_name": "default", "job": 35016, "event": "table_file_creation", "file_number": 278964, "file_size": 37663500, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4155809461, "largest_seqno": 4239498632, "table_properties": {"data_size": 31702352, "index_size": 1756544, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25848340, "raw_average_key_size": 28, "raw_value_size": 108369450, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923155, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364241, "oldest_key_time": 0, "file_creation_time": 1732364241, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278964, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:17:26.763161 139957164807744 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278965.sst) /dev/shm/ToplingTemp/Topling-177538-kpfkVN 2024/11/23-20:17:27.922560 139957164807744 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278965.sst): first pass time = 1.16's, 115.769'MB/sec 2024/11/23-20:17:29.864030 139957164807744 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278965.sst): old prealloc_size = 147639500, real_size = 34479104 2024/11/23-20:17:29.864566 139957164807744 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278965.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928420 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0325 GB avg-val = 116.57 avg-zval = 34.99 2024/11/23-20:17:29.865348 139957164807744 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278965.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:17:30.916477 139957164807744 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278965.sst): second pass time = 1.94's, 55.764'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.05's, 36.439'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3409.806'MB/sec (index lex order gen) rebuild zvType time = 0.92's, 0.252'MB/sec write SST data time = 0.13's, 289.834'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 37284.678'MB/sec zip my value time = 1.94's, unzip length = 0.108'GB zip my value throughput = 55.764'MB/sec zip pipeline throughput = 130.126'MB/sec entries = 928420 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.57 avg-zval = 34.99 usrkeys = 928420 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.57 avg-zval = 34.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928420 bytes = 3017376 } seq expand size = 6498940 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.5147 value = 3.3319 dict = 1.00 all = 3.4998 } Zip/UnZip{ index = 0.0951 value = 0.3001 dict = 1.00 all = 0.2857 } ---------------------------- total value len = 9.307660 GB avg = 0.166 KB (by entry num) total key len = 1.544595 GB avg = 0.028 KB total ukey len = 1.072003 GB avg = 0.015 KB total ukey num = 0.069256663 Billion total entry num = 0.056064392 Billion write speed all = 0.265085485 MB/sec (with seq num) write speed all = 0.254129711 MB/sec (without seq num) 2024/11/23-20:17:30.969081 139957164807744 [compaction_job.cc:1948] [default] [JOB 35016] Generated table #278965: 928420 keys, 38349690 bytes, temperature: kWarm 2024/11/23-20:17:30.969149 139957164807744 EVENT_LOG_v1 {"time_micros": 1732364250969114, "cf_name": "default", "job": 35016, "event": "table_file_creation", "file_number": 278965, "file_size": 38349690, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4191256266, "largest_seqno": 4239498563, "table_properties": {"data_size": 32480880, "index_size": 1765952, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25995760, "raw_average_key_size": 28, "raw_value_size": 108222052, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928420, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364246, "oldest_key_time": 0, "file_creation_time": 1732364246, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278965, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:17:31.005836 139957164807744 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278966.sst) /dev/shm/ToplingTemp/Topling-177538-3tpWzO 2024/11/23-20:17:31.961621 139957164807744 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278966.sst): first pass time = 0.96's, 140.433'MB/sec 2024/11/23-20:17:33.576886 139957164807744 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278966.sst): old prealloc_size = 147639500, real_size = 35293744 2024/11/23-20:17:33.577489 139957164807744 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278966.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933077 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0333 GB avg-val = 115.84 avg-zval = 35.67 2024/11/23-20:17:33.578289 139957164807744 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278966.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:17:34.586905 139957164807744 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278966.sst): second pass time = 1.61's, 66.951'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.01's, 39.139'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3057.834'MB/sec (index lex order gen) rebuild zvType time = 0.88's, 0.265'MB/sec write SST data time = 0.13's, 306.584'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32652.669'MB/sec zip my value time = 1.61's, unzip length = 0.108'GB zip my value throughput = 66.951'MB/sec zip pipeline throughput = 66.951'MB/sec entries = 933077 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.84 avg-zval = 35.67 usrkeys = 933077 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.84 avg-zval = 35.67 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933077 bytes = 3382416 } seq expand size = 6531539 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0333 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.5118 value = 3.2475 dict = 1.00 all = 3.3953 } Zip/UnZip{ index = 0.0951 value = 0.3079 dict = 1.00 all = 0.2945 } ---------------------------- total value len = 9.422283 GB avg = 0.165 KB (by entry num) total key len = 1.570722 GB avg = 0.028 KB total ukey len = 1.090664 GB avg = 0.016 KB total ukey num = 0.070189740 Billion total entry num = 0.056997469 Billion write speed all = 0.268499463 MB/sec (with seq num) write speed all = 0.257362352 MB/sec (without seq num) 2024/11/23-20:17:34.638989 139957164807744 [compaction_job.cc:1948] [default] [JOB 35016] Generated table #278966: 933077 keys, 39530294 bytes, temperature: kWarm 2024/11/23-20:17:34.639047 139957164807744 EVENT_LOG_v1 {"time_micros": 1732364254639013, "cf_name": "default", "job": 35016, "event": "table_file_creation", "file_number": 278966, "file_size": 39530294, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3736072251, "largest_seqno": 4239498290, "table_properties": {"data_size": 33285024, "index_size": 1775296, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26126156, "raw_average_key_size": 28, "raw_value_size": 108091662, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933077, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364250, "oldest_key_time": 0, "file_creation_time": 1732364250, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278966, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:17:34.682489 139957164807744 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278967.sst) /dev/shm/ToplingTemp/Topling-177538-FyjN3M 2024/11/23-20:17:35.719704 139957164807744 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278967.sst): first pass time = 1.04's, 129.407'MB/sec 2024/11/23-20:17:37.268207 139957164807744 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278967.sst): old prealloc_size = 147639500, real_size = 36735056 2024/11/23-20:17:37.268929 139957164807744 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278967.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932449 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0347 GB avg-val = 115.94 avg-zval = 37.22 2024/11/23-20:17:37.269680 139957164807744 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278967.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:17:38.220696 139957164807744 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278967.sst): second pass time = 1.55's, 69.883'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 42.082'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2621.739'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.294'MB/sec write SST data time = 0.16's, 253.538'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36566.844'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 69.883'MB/sec zip pipeline throughput = 69.883'MB/sec entries = 932449 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.94 avg-zval = 37.22 usrkeys = 932449 avg-key = 20.00 avg-zkey = 1.93 avg-val = 120.64 avg-zval = 37.22 TagRS{ kinds = RS_Key0_TagN bytes = 145800 } TagArray{ size = 625657 bytes = 2346224 } seq expand size = 4379599 multi value expand size = 0 cnt WriteAppend = 22 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0347 GB dict = 1.08 MB all = 0.0401 GB } UnZip/Zip{ index = 10.3768 value = 3.1151 dict = 1.00 all = 3.3486 } Zip/UnZip{ index = 0.0964 value = 0.3210 dict = 1.00 all = 0.2986 } ---------------------------- total value len = 9.534772 GB avg = 0.165 KB (by entry num) total key len = 1.596830 GB avg = 0.028 KB total ukey len = 1.109313 GB avg = 0.016 KB total ukey num = 0.071122189 Billion total entry num = 0.057929918 Billion write speed all = 0.271860518 MB/sec (with seq num) write speed all = 0.260542214 MB/sec (without seq num) 2024/11/23-20:17:38.273221 139957164807744 [compaction_job.cc:1948] [default] [JOB 35016] Generated table #278967: 932449 keys, 40081618 bytes, temperature: kWarm 2024/11/23-20:17:38.273297 139957164807744 EVENT_LOG_v1 {"time_micros": 1732364258273262, "cf_name": "default", "job": 35016, "event": "table_file_creation", "file_number": 278967, "file_size": 40081618, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776165, "table_properties": {"data_size": 34704576, "index_size": 1797184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26108572, "raw_average_key_size": 28, "raw_value_size": 108109286, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932449, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364254, "oldest_key_time": 0, "file_creation_time": 1732364254, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278967, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:17:38.309916 139957164807744 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278968.sst) /dev/shm/ToplingTemp/Topling-177538-qTUlJO 2024/11/23-20:17:39.471737 139957164807744 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278968.sst): first pass time = 1.16's, 115.527'MB/sec 2024/11/23-20:17:40.927156 139957164807744 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278968.sst): old prealloc_size = 147639500, real_size = 35590464 2024/11/23-20:17:40.927888 139957164807744 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278968.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928960 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0336 GB avg-val = 116.48 avg-zval = 36.15 2024/11/23-20:17:40.928767 139957164807744 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278968.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:17:41.920539 139957164807744 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278968.sst): second pass time = 1.45's, 74.391'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.99's, 40.077'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2506.641'MB/sec (index lex order gen) rebuild zvType time = 0.84's, 0.277'MB/sec write SST data time = 0.16's, 255.921'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 36206.850'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.391'MB/sec zip pipeline throughput = 74.391'MB/sec entries = 928960 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.48 avg-zval = 36.15 usrkeys = 928960 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.48 avg-zval = 36.15 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928960 bytes = 3367488 } seq expand size = 6502720 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0336 GB dict = 1.08 MB all = 0.0398 GB } UnZip/Zip{ index = 10.4436 value = 3.2225 dict = 1.00 all = 3.3714 } Zip/UnZip{ index = 0.0958 value = 0.3103 dict = 1.00 all = 0.2966 } ---------------------------- total value len = 9.649482 GB avg = 0.164 KB (by entry num) total key len = 1.622841 GB avg = 0.028 KB total ukey len = 1.127892 GB avg = 0.016 KB total ukey num = 0.072051149 Billion total entry num = 0.058858878 Billion write speed all = 0.275272377 MB/sec (with seq num) write speed all = 0.263773613 MB/sec (without seq num) 2024/11/23-20:17:41.974270 139957164807744 [compaction_job.cc:1948] [default] [JOB 35016] Generated table #278968: 928960 keys, 39810890 bytes, temperature: kWarm 2024/11/23-20:17:41.974338 139957164807744 EVENT_LOG_v1 {"time_micros": 1732364261974304, "cf_name": "default", "job": 35016, "event": "table_file_creation", "file_number": 278968, "file_size": 39810890, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775967, "table_properties": {"data_size": 33579056, "index_size": 1779008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26010880, "raw_average_key_size": 28, "raw_value_size": 108206852, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928960, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364258, "oldest_key_time": 0, "file_creation_time": 1732364258, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278968, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:17:42.012412 139957164807744 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278969.sst) /dev/shm/ToplingTemp/Topling-177538-8xdsYK 2024/11/23-20:17:43.209836 139957164807744 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278969.sst): first pass time = 1.20's, 112.093'MB/sec 2024/11/23-20:17:45.015834 139957164807744 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278969.sst): old prealloc_size = 147639500, real_size = 32524736 2024/11/23-20:17:45.016430 139957164807744 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278969.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927895 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0305 GB avg-val = 116.65 avg-zval = 32.90 2024/11/23-20:17:45.017276 139957164807744 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278969.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:17:45.977898 139957164807744 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278969.sst): second pass time = 1.81's, 59.960'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 38.313'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3122.716'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.313'MB/sec write SST data time = 0.22's, 166.679'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32961.362'MB/sec zip my value time = 1.81's, unzip length = 0.108'GB zip my value throughput = 59.960'MB/sec zip pipeline throughput = 132.879'MB/sec entries = 927895 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.65 avg-zval = 32.90 usrkeys = 927895 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.65 avg-zval = 32.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927895 bytes = 3479616 } seq expand size = 6495265 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5141 value = 3.5455 dict = 1.00 all = 3.6415 } Zip/UnZip{ index = 0.0951 value = 0.2820 dict = 1.00 all = 0.2746 } ---------------------------- total value len = 9.764214 GB avg = 0.163 KB (by entry num) total key len = 1.648822 GB avg = 0.028 KB total ukey len = 1.146450 GB avg = 0.016 KB total ukey num = 0.072979044 Billion total entry num = 0.059786773 Billion write speed all = 0.278681005 MB/sec (with seq num) write speed all = 0.267002123 MB/sec (without seq num) 2024/11/23-20:17:46.029827 139957164807744 [compaction_job.cc:1948] [default] [JOB 35016] Generated table #278969: 927895 keys, 36858268 bytes, temperature: kWarm 2024/11/23-20:17:46.029918 139957164807744 EVENT_LOG_v1 {"time_micros": 1732364266029870, "cf_name": "default", "job": 35016, "event": "table_file_creation", "file_number": 278969, "file_size": 36858268, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 4012776021, "table_properties": {"data_size": 30527536, "index_size": 1765056, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25981060, "raw_average_key_size": 28, "raw_value_size": 108236702, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927895, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364261, "oldest_key_time": 0, "file_creation_time": 1732364261, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278969, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:17:46.098856 139957164807744 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278970.sst) /dev/shm/ToplingTemp/Topling-177538-eeGA6M 2024/11/23-20:17:47.219764 139957164807744 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278970.sst): first pass time = 1.12's, 119.746'MB/sec 2024/11/23-20:17:48.759944 139957164807744 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278970.sst): old prealloc_size = 147639500, real_size = 34206032 2024/11/23-20:17:48.760512 139957164807744 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278970.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928688 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0322 GB avg-val = 116.52 avg-zval = 34.67 2024/11/23-20:17:48.761306 139957164807744 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278970.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:17:49.870422 139957164807744 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278970.sst): second pass time = 1.54's, 70.299'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.11's, 34.224'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3382.704'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.266'MB/sec write SST data time = 0.24's, 161.103'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35885.137'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.299'MB/sec zip pipeline throughput = 70.299'MB/sec entries = 928688 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.52 avg-zval = 34.67 usrkeys = 928688 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.14 avg-zval = 34.67 TagRS{ kinds = RS_Key0_TagN bytes = 145200 } TagArray{ size = 745691 bytes = 2796352 } seq expand size = 5219837 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0322 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.4454 value = 3.3612 dict = 1.00 all = 3.5316 } Zip/UnZip{ index = 0.0957 value = 0.2975 dict = 1.00 all = 0.2832 } ---------------------------- total value len = 9.877648 GB avg = 0.163 KB (by entry num) total key len = 1.674825 GB avg = 0.028 KB total ukey len = 1.165024 GB avg = 0.016 KB total ukey num = 0.073907732 Billion total entry num = 0.060715461 Billion write speed all = 0.282058954 MB/sec (with seq num) write speed all = 0.270199787 MB/sec (without seq num) 2024/11/23-20:17:49.919089 139957164807744 [compaction_job.cc:1948] [default] [JOB 35016] Generated table #278970: 928688 keys, 38004592 bytes, temperature: kWarm 2024/11/23-20:17:49.919146 139957164807744 EVENT_LOG_v1 {"time_micros": 1732364269919113, "cf_name": "default", "job": 35016, "event": "table_file_creation", "file_number": 278970, "file_size": 38004592, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776210, "table_properties": {"data_size": 32195520, "index_size": 1778176, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26003264, "raw_average_key_size": 28, "raw_value_size": 108214532, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928688, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364266, "oldest_key_time": 0, "file_creation_time": 1732364266, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278970, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:17:49.954479 139957164807744 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278971.sst) /dev/shm/ToplingTemp/Topling-177538-hBGwIO 2024/11/23-20:17:50.864485 139957164807744 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278971.sst): first pass time = 0.91's, 147.497'MB/sec 2024/11/23-20:17:52.378247 139957164807744 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278971.sst): old prealloc_size = 147639500, real_size = 31211952 2024/11/23-20:17:52.378776 139957164807744 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278971.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933457 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0292 GB avg-val = 115.79 avg-zval = 31.28 2024/11/23-20:17:52.379642 139957164807744 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278971.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:17:53.419779 139957164807744 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278971.sst): second pass time = 1.51's, 71.452'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.04's, 34.033'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3512.706'MB/sec (index lex order gen) rebuild zvType time = 0.86's, 0.272'MB/sec write SST data time = 0.18's, 193.402'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31246.319'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.452'MB/sec zip pipeline throughput = 71.452'MB/sec entries = 933457 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.79 avg-zval = 31.28 usrkeys = 933457 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.79 avg-zval = 31.28 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933457 bytes = 3383792 } seq expand size = 6534199 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0354 GB } UnZip/Zip{ index = 10.5138 value = 3.7011 dict = 1.00 all = 3.7865 } Zip/UnZip{ index = 0.0951 value = 0.2702 dict = 1.00 all = 0.2641 } ---------------------------- total value len = 9.992263 GB avg = 0.162 KB (by entry num) total key len = 1.700962 GB avg = 0.028 KB total ukey len = 1.183693 GB avg = 0.016 KB total ukey num = 0.074841189 Billion total entry num = 0.061648918 Billion write speed all = 0.285470741 MB/sec (with seq num) write speed all = 0.273430291 MB/sec (without seq num) 2024/11/23-20:17:53.465927 139957164807744 [compaction_job.cc:1948] [default] [JOB 35016] Generated table #278971: 933457 keys, 35446270 bytes, temperature: kWarm 2024/11/23-20:17:53.466012 139957164807744 EVENT_LOG_v1 {"time_micros": 1732364273465957, "cf_name": "default", "job": 35016, "event": "table_file_creation", "file_number": 278971, "file_size": 35446270, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776228, "table_properties": {"data_size": 29202720, "index_size": 1775680, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26136796, "raw_average_key_size": 28, "raw_value_size": 108081050, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933457, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364269, "oldest_key_time": 0, "file_creation_time": 1732364269, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278971, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:17:53.506614 139957164807744 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278972.sst) /dev/shm/ToplingTemp/Topling-177538-H3AbmM 2024/11/23-20:17:54.423730 139957164807744 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278972.sst): first pass time = 0.92's, 146.356'MB/sec 2024/11/23-20:17:55.958602 139957164807744 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278972.sst): old prealloc_size = 147639500, real_size = 33096464 2024/11/23-20:17:55.959146 139957164807744 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278972.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931033 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0311 GB avg-val = 116.16 avg-zval = 33.38 2024/11/23-20:17:55.959942 139957164807744 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278972.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:17:56.810651 139957164807744 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278972.sst): second pass time = 1.53's, 70.513'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.805'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3396.863'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 287.460'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 36468.790'MB/sec zip my value time = 1.53's, unzip length = 0.108'GB zip my value throughput = 70.513'MB/sec zip pipeline throughput = 70.513'MB/sec entries = 931033 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.16 avg-zval = 33.38 usrkeys = 931033 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.16 avg-zval = 33.38 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931033 bytes = 3375008 } seq expand size = 6517231 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4458 value = 3.4796 dict = 1.00 all = 3.5961 } Zip/UnZip{ index = 0.0957 value = 0.2874 dict = 1.00 all = 0.2781 } ---------------------------- total value len = 10.106930 GB avg = 0.162 KB (by entry num) total key len = 1.727031 GB avg = 0.028 KB total ukey len = 1.202314 GB avg = 0.016 KB total ukey num = 0.075772222 Billion total entry num = 0.062579951 Billion write speed all = 0.288882641 MB/sec (with seq num) write speed all = 0.276661366 MB/sec (without seq num) 2024/11/23-20:17:56.858672 139957164807744 [compaction_job.cc:1948] [default] [JOB 35016] Generated table #278972: 931033 keys, 37323152 bytes, temperature: kWarm 2024/11/23-20:17:56.858740 139957164807744 EVENT_LOG_v1 {"time_micros": 1732364276858704, "cf_name": "default", "job": 35016, "event": "table_file_creation", "file_number": 278972, "file_size": 37323152, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776268, "table_properties": {"data_size": 31080960, "index_size": 1782592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26068924, "raw_average_key_size": 28, "raw_value_size": 108148938, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931033, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364273, "oldest_key_time": 0, "file_creation_time": 1732364273, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278972, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:17:56.901871 139957164807744 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278973.sst) /dev/shm/ToplingTemp/Topling-177538-H68JyL 2024/11/23-20:17:57.958496 139957164807744 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278973.sst): first pass time = 1.06's, 127.031'MB/sec 2024/11/23-20:17:59.323289 139957164807744 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278973.sst): old prealloc_size = 147639500, real_size = 34491168 2024/11/23-20:17:59.323844 139957164807744 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278973.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923909 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0325 GB avg-val = 117.27 avg-zval = 35.17 2024/11/23-20:17:59.324717 139957164807744 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278973.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:18:00.171046 139957164807744 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278973.sst): second pass time = 1.36's, 79.439'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.784'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3340.180'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 301.382'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 34648.375'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.439'MB/sec zip pipeline throughput = 79.439'MB/sec entries = 923909 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.27 avg-zval = 35.17 usrkeys = 923909 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.27 avg-zval = 35.17 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923909 bytes = 3464672 } seq expand size = 6467363 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0388 GB } UnZip/Zip{ index = 10.4416 value = 3.3348 dict = 1.00 all = 3.4581 } Zip/UnZip{ index = 0.0958 value = 0.2999 dict = 1.00 all = 0.2892 } ---------------------------- total value len = 10.221745 GB avg = 0.161 KB (by entry num) total key len = 1.752901 GB avg = 0.028 KB total ukey len = 1.220792 GB avg = 0.016 KB total ukey num = 0.076696131 Billion total entry num = 0.063503860 Billion write speed all = 0.292292974 MB/sec (with seq num) write speed all = 0.279892285 MB/sec (without seq num) 2024/11/23-20:18:00.219781 139957164807744 [compaction_job.cc:1948] [default] [JOB 35016] Generated table #278973: 923909 keys, 38812882 bytes, temperature: kWarm 2024/11/23-20:18:00.219854 139957164807744 EVENT_LOG_v1 {"time_micros": 1732364280219816, "cf_name": "default", "job": 35016, "event": "table_file_creation", "file_number": 278973, "file_size": 38812882, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012774796, "table_properties": {"data_size": 32490320, "index_size": 1769664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25869452, "raw_average_key_size": 28, "raw_value_size": 108348310, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923909, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364276, "oldest_key_time": 0, "file_creation_time": 1732364276, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278973, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:18:00.259579 139957164807744 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278974.sst) /dev/shm/ToplingTemp/Topling-177538-EncVFM 2024/11/23-20:18:01.122169 139957164807744 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278974.sst): first pass time = 0.86's, 155.607'MB/sec 2024/11/23-20:18:02.476026 139957164807744 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278974.sst): old prealloc_size = 147639500, real_size = 32851456 2024/11/23-20:18:02.476609 139957164807744 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278974.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933794 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 115.73 avg-zval = 33.03 2024/11/23-20:18:02.477449 139957164807744 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278974.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:18:03.340193 139957164807744 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278974.sst): second pass time = 1.35's, 79.875'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.918'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3189.296'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 283.981'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32796.289'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 79.875'MB/sec zip pipeline throughput = 79.875'MB/sec entries = 933794 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.73 avg-zval = 33.03 usrkeys = 933794 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.73 avg-zval = 33.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933794 bytes = 3385024 } seq expand size = 6536558 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5138 value = 3.5041 dict = 1.00 all = 3.6190 } Zip/UnZip{ index = 0.0951 value = 0.2854 dict = 1.00 all = 0.2763 } ---------------------------- total value len = 10.336353 GB avg = 0.160 KB (by entry num) total key len = 1.779047 GB avg = 0.028 KB total ukey len = 1.239468 GB avg = 0.016 KB total ukey num = 0.077629925 Billion total entry num = 0.064437654 Billion write speed all = 0.295705819 MB/sec (with seq num) write speed all = 0.283123757 MB/sec (without seq num) 2024/11/23-20:18:03.388786 139957164807744 [compaction_job.cc:1948] [default] [JOB 35016] Generated table #278974: 933794 keys, 37087154 bytes, temperature: kWarm 2024/11/23-20:18:03.388860 139957164807744 EVENT_LOG_v1 {"time_micros": 1732364283388823, "cf_name": "default", "job": 35016, "event": "table_file_creation", "file_number": 278974, "file_size": 37087154, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776316, "table_properties": {"data_size": 30841520, "index_size": 1776320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26146232, "raw_average_key_size": 28, "raw_value_size": 108071628, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933794, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364280, "oldest_key_time": 0, "file_creation_time": 1732364280, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278974, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:18:03.425315 139957164807744 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278975.sst) /dev/shm/ToplingTemp/Topling-177538-TyB6eP 2024/11/23-20:18:04.263243 139957164807744 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278975.sst): first pass time = 0.84's, 160.187'MB/sec 2024/11/23-20:18:05.572293 139957164807744 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278975.sst): old prealloc_size = 147639500, real_size = 33418976 2024/11/23-20:18:05.572850 139957164807744 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278975.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923876 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 117.28 avg-zval = 34.02 2024/11/23-20:18:05.573705 139957164807744 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278975.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:18:06.419942 139957164807744 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278975.sst): second pass time = 1.31's, 82.819'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.525'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3302.822'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.12's, 302.019'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 28183.144'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.819'MB/sec zip pipeline throughput = 82.819'MB/sec entries = 923876 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.28 avg-zval = 34.02 usrkeys = 923876 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.28 avg-zval = 34.02 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923876 bytes = 3464544 } seq expand size = 6467132 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5124 value = 3.4473 dict = 1.00 all = 3.5563 } Zip/UnZip{ index = 0.0951 value = 0.2901 dict = 1.00 all = 0.2812 } ---------------------------- total value len = 10.451170 GB avg = 0.160 KB (by entry num) total key len = 1.804915 GB avg = 0.028 KB total ukey len = 1.257945 GB avg = 0.016 KB total ukey num = 0.078553801 Billion total entry num = 0.065361530 Billion write speed all = 0.299117093 MB/sec (with seq num) write speed all = 0.286355595 MB/sec (without seq num) 2024/11/23-20:18:06.466339 139957164807744 [compaction_job.cc:1948] [default] [JOB 35016] Generated table #278975: 923876 keys, 37740750 bytes, temperature: kWarm 2024/11/23-20:18:06.466409 139957164807744 EVENT_LOG_v1 {"time_micros": 1732364286466374, "cf_name": "default", "job": 35016, "event": "table_file_creation", "file_number": 278975, "file_size": 37740750, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776344, "table_properties": {"data_size": 31430096, "index_size": 1757696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25868528, "raw_average_key_size": 28, "raw_value_size": 108349288, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923876, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364283, "oldest_key_time": 0, "file_creation_time": 1732364283, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278975, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:18:06.500901 139957164807744 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278976.sst) /dev/shm/ToplingTemp/Topling-177538-qbiswM 2024/11/23-20:18:07.436992 139957164807744 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278976.sst): first pass time = 0.94's, 143.387'MB/sec 2024/11/23-20:18:08.765263 139957164807744 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278976.sst): old prealloc_size = 147639500, real_size = 33152768 2024/11/23-20:18:08.765775 139957164807744 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278976.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927353 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 116.73 avg-zval = 33.57 2024/11/23-20:18:08.766643 139957164807744 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278976.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:18:09.616007 139957164807744 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278976.sst): second pass time = 1.33's, 81.545'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.062'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3666.193'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.12's, 301.243'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33155.565'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.545'MB/sec zip pipeline throughput = 81.545'MB/sec entries = 927353 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.73 avg-zval = 33.57 usrkeys = 927353 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.73 avg-zval = 33.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927353 bytes = 3477584 } seq expand size = 6491471 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.3762 value = 3.4770 dict = 1.00 all = 3.5806 } Zip/UnZip{ index = 0.0964 value = 0.2876 dict = 1.00 all = 0.2793 } ---------------------------- total value len = 10.565913 GB avg = 0.159 KB (by entry num) total key len = 1.830881 GB avg = 0.028 KB total ukey len = 1.276492 GB avg = 0.016 KB total ukey num = 0.079481154 Billion total entry num = 0.066288883 Billion write speed all = 0.302527591 MB/sec (with seq num) write speed all = 0.289586041 MB/sec (without seq num) 2024/11/23-20:18:09.662212 139957164807744 [compaction_job.cc:1948] [default] [JOB 35016] Generated table #278976: 927353 keys, 37484510 bytes, temperature: kWarm 2024/11/23-20:18:09.662291 139957164807744 EVENT_LOG_v1 {"time_micros": 1732364289662254, "cf_name": "default", "job": 35016, "event": "table_file_creation", "file_number": 278976, "file_size": 37484510, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012771944, "table_properties": {"data_size": 31133296, "index_size": 1787456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25965884, "raw_average_key_size": 28, "raw_value_size": 108251978, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927353, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364286, "oldest_key_time": 0, "file_creation_time": 1732364286, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278976, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:18:09.709288 139957164807744 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278977.sst) /dev/shm/ToplingTemp/Topling-177538-CJkuSO 2024/11/23-20:18:10.567215 139957164807744 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278977.sst): first pass time = 0.86's, 156.452'MB/sec 2024/11/23-20:18:12.010724 139957164807744 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278977.sst): old prealloc_size = 147639500, real_size = 35594880 2024/11/23-20:18:12.011325 139957164807744 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278977.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929660 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0336 GB avg-val = 116.37 avg-zval = 36.14 2024/11/23-20:18:12.011703 139957164807744 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278977.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:18:12.763164 139957164807744 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278977.sst): second pass time = 1.44's, 74.986'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.75's, 50.187'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3054.534'MB/sec (index lex order gen) rebuild zvType time = 0.63's, 0.370'MB/sec write SST data time = 0.12's, 304.762'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34917.702'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 74.986'MB/sec zip pipeline throughput = 74.986'MB/sec entries = 929660 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.37 avg-zval = 36.14 usrkeys = 929660 avg-key = 20.00 avg-zkey = 1.90 avg-val = 119.07 avg-zval = 36.14 TagRS{ kinds = RS_Key0_TagN bytes = 145360 } TagArray{ size = 358032 bytes = 1163616 } seq expand size = 2506224 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0336 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.5116 value = 3.2205 dict = 1.00 all = 3.5544 } Zip/UnZip{ index = 0.0951 value = 0.3105 dict = 1.00 all = 0.2813 } ---------------------------- total value len = 10.676607 GB avg = 0.159 KB (by entry num) total key len = 1.856912 GB avg = 0.028 KB total ukey len = 1.295086 GB avg = 0.016 KB total ukey num = 0.080410814 Billion total entry num = 0.067218543 Billion write speed all = 0.305840674 MB/sec (with seq num) write speed all = 0.292718635 MB/sec (without seq num) 2024/11/23-20:18:12.811785 139957164807744 [compaction_job.cc:1948] [default] [JOB 35016] Generated table #278977: 929660 keys, 37761311 bytes, temperature: kWarm 2024/11/23-20:18:12.811848 139957164807744 EVENT_LOG_v1 {"time_micros": 1732364292811815, "cf_name": "default", "job": 35016, "event": "table_file_creation", "file_number": 278977, "file_size": 37761311, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 33593456, "index_size": 1768832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26030480, "raw_average_key_size": 28, "raw_value_size": 108187360, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929660, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364289, "oldest_key_time": 0, "file_creation_time": 1732364289, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278977, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:18:12.849630 139957164807744 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278978.sst) /dev/shm/ToplingTemp/Topling-177538-6XinlL 2024/11/23-20:18:13.872529 139957164807744 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278978.sst): first pass time = 1.02's, 131.218'MB/sec 2024/11/23-20:18:15.373868 139957164807744 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278978.sst): old prealloc_size = 147639500, real_size = 37154240 2024/11/23-20:18:15.374474 139957164807744 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278978.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926452 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0351 GB avg-val = 116.87 avg-zval = 37.94 2024/11/23-20:18:15.374756 139957164807744 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278978.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:18:16.088156 139957164807744 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278978.sst): second pass time = 1.50's, 72.183'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.71's, 54.453'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3048.373'MB/sec (index lex order gen) rebuild zvType time = 0.59's, 0.392'MB/sec write SST data time = 0.12's, 315.312'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33494.155'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 72.183'MB/sec zip pipeline throughput = 72.183'MB/sec entries = 926452 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.87 avg-zval = 37.94 usrkeys = 926452 avg-key = 20.00 avg-zkey = 1.91 avg-val = 118.36 avg-zval = 37.94 TagRS{ kinds = RS_Key0_TagN bytes = 144840 } TagArray{ size = 196487 bytes = 736848 } seq expand size = 1375409 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0351 GB dict = 1.08 MB all = 0.0389 GB } UnZip/Zip{ index = 10.4469 value = 3.0805 dict = 1.00 all = 3.4509 } Zip/UnZip{ index = 0.0957 value = 0.3246 dict = 1.00 all = 0.2898 } ---------------------------- total value len = 10.786259 GB avg = 0.158 KB (by entry num) total key len = 1.882852 GB avg = 0.028 KB total ukey len = 1.313615 GB avg = 0.016 KB total ukey num = 0.081337266 Billion total entry num = 0.068144995 Billion write speed all = 0.309124313 MB/sec (with seq num) write speed all = 0.295822497 MB/sec (without seq num) 2024/11/23-20:18:16.136289 139957164807744 [compaction_job.cc:1948] [default] [JOB 35016] Generated table #278978: 926452 keys, 38894059 bytes, temperature: kWarm 2024/11/23-20:18:16.136359 139957164807744 EVENT_LOG_v1 {"time_micros": 1732364296136324, "cf_name": "default", "job": 35016, "event": "table_file_creation", "file_number": 278978, "file_size": 38894059, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4020965195, "table_properties": {"data_size": 35148784, "index_size": 1773632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25940656, "raw_average_key_size": 28, "raw_value_size": 108277104, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926452, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364292, "oldest_key_time": 0, "file_creation_time": 1732364292, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278978, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:18:16.175639 139957164807744 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278979.sst) /dev/shm/ToplingTemp/Topling-177538-LIlEMO 2024/11/23-20:18:17.203359 139957164807744 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278979.sst): first pass time = 1.03's, 130.603'MB/sec 2024/11/23-20:18:18.544491 139957164807744 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278979.sst): old prealloc_size = 147639500, real_size = 34608160 2024/11/23-20:18:18.545053 139957164807744 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278979.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928212 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0326 GB avg-val = 116.60 avg-zval = 35.12 2024/11/23-20:18:18.545855 139957164807744 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278979.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:18:19.383267 139957164807744 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278979.sst): second pass time = 1.34's, 80.749'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.153'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3262.824'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.12's, 311.356'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33484.973'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.749'MB/sec zip pipeline throughput = 80.749'MB/sec entries = 928212 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.60 avg-zval = 35.12 usrkeys = 928212 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.60 avg-zval = 35.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928212 bytes = 3248752 } seq expand size = 6497484 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0326 GB dict = 1.08 MB all = 0.0387 GB } UnZip/Zip{ index = 10.4453 value = 3.3200 dict = 1.00 all = 3.4672 } Zip/UnZip{ index = 0.0957 value = 0.3012 dict = 1.00 all = 0.2884 } ---------------------------- total value len = 10.900985 GB avg = 0.158 KB (by entry num) total key len = 1.908842 GB avg = 0.028 KB total ukey len = 1.332179 GB avg = 0.016 KB total ukey num = 0.082265478 Billion total entry num = 0.069073207 Billion write speed all = 0.312532616 MB/sec (with seq num) write speed all = 0.299050698 MB/sec (without seq num) 2024/11/23-20:18:19.430481 139957164807744 [compaction_job.cc:1948] [default] [JOB 35016] Generated table #278979: 928212 keys, 38710392 bytes, temperature: kWarm 2024/11/23-20:18:19.430549 139957164807744 EVENT_LOG_v1 {"time_micros": 1732364299430515, "cf_name": "default", "job": 35016, "event": "table_file_creation", "file_number": 278979, "file_size": 38710392, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4020965196, "largest_seqno": 4239498428, "table_properties": {"data_size": 32598672, "index_size": 1777280, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25989936, "raw_average_key_size": 28, "raw_value_size": 108227864, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928212, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364296, "oldest_key_time": 0, "file_creation_time": 1732364296, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278979, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:18:19.466061 139957164807744 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278980.sst) /dev/shm/ToplingTemp/Topling-177538-h4oddP 2024/11/23-20:18:19.549463 139957164807744 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278980.sst): first pass time = 0.08's, 237.636'MB/sec 2024/11/23-20:18:19.732769 139957164807744 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278980.sst): old prealloc_size = 147639500, real_size = 5835504 2024/11/23-20:18:19.732904 139957164807744 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278980.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 135928 prefix = 4 raw-key = 0.0022 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0160 GB zip-val = 0.0055 GB avg-val = 117.74 avg-zval = 40.75 2024/11/23-20:18:19.733082 139957164807744 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278980.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.020 GB 2024/11/23-20:18:19.853470 139957164807744 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00/278980.sst): second pass time = 0.18's, 87.621'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.12's, 53.135'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2347.612'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.336'MB/sec write SST data time = 0.02's, 332.028'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.168'MB, throughput = 4272.239'MB/sec zip my value time = 0.18's, unzip length = 0.016'GB zip my value throughput = 87.621'MB/sec zip pipeline throughput = 87.621'MB/sec entries = 135928 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.74 avg-zval = 40.75 usrkeys = 135928 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.74 avg-zval = 40.75 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 135928 bytes = 441776 } seq expand size = 951496 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0027 GB value = 0.0160 GB dict = 0.17 MB all = 0.0198 GB } __ZipSize{ index = 0.0003 GB value = 0.0055 GB dict = 0.17 MB all = 0.0064 GB } UnZip/Zip{ index = 10.3908 value = 2.8891 dict = 1.00 all = 3.0894 } Zip/UnZip{ index = 0.0962 value = 0.3461 dict = 1.00 all = 0.3237 } ---------------------------- total value len = 10.917941 GB avg = 0.158 KB (by entry num) total key len = 1.912648 GB avg = 0.028 KB total ukey len = 1.334897 GB avg = 0.016 KB total ukey num = 0.082401406 Billion total entry num = 0.069209135 Billion write speed all = 0.313035573 MB/sec (with seq num) write speed all = 0.299527278 MB/sec (without seq num) 2024/11/23-20:18:19.869620 139957164807744 [compaction_job.cc:1948] [default] [JOB 35016] Generated table #278980: 135928 keys, 6412415 bytes, temperature: kWarm 2024/11/23-20:18:19.869677 139957164807744 EVENT_LOG_v1 {"time_micros": 1732364299869647, "cf_name": "default", "job": 35016, "event": "table_file_creation", "file_number": 278980, "file_size": 6412415, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4203102636, "largest_seqno": 4238000034, "table_properties": {"data_size": 5539680, "index_size": 261632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3805984, "raw_average_key_size": 28, "raw_value_size": 16004484, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 135928, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364299, "oldest_key_time": 0, "file_creation_time": 1732364299, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278980, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:18:19.881697 139957164807744 [compaction_job.cc:703] job-35016: subcompact[0], size: 0.691904 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35016/att-00 [278962,278963,278964,278965,278966,278967,278968,278969,278970,278971,278972,278973,278974,278975,278976,278977,278978,278979,278980] 2024/11/23-20:18:19.994153 139957164807744 (Original Log Time 2024/11/23-20:17:11.673411) EVENT_LOG_v1 {"time_micros": 1732364231673344, "job": 35016, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [278955], "files_L3": [278893, 278894, 278895, 278896, 278897, 278898, 278899, 278900, 278901, 278902, 278903, 278904, 278905, 278906, 278907, 278908, 278909, 278918], "score": 0.991378, "input_data_size": 712321378, "oldest_snapshot_seqno": 4239518722} 2024/11/23-20:18:19.994157 139957164807744 (Original Log Time 2024/11/23-20:18:19.992633) [compaction_job.cc:2022] [default] [JOB 35016] Compacted 1@2 + 18@3 files to L3 => 691903557 bytes 2024/11/23-20:18:19.994158 139957164807744 (Original Log Time 2024/11/23-20:18:19.994090) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 19 0 0 0] max score 0.03, MB/sec: 10.4 rd, 10.1 wr, level 3, files in(1, 18) out(19 +0 blob) MB in(27.1, 652.2 +0.0 blob) out(659.9 +0.0 blob), read-write-amplify(49.4) write-amplify(24.3) OK, records in: 16947370, records dropped: 96399 output_compression: Snappy 2024/11/23-20:18:19.994165 139957164807744 (Original Log Time 2024/11/23-20:18:19.994133) EVENT_LOG_v1 {"time_micros": 1732364299994113, "cf": "default", "job": 35016, "event": "compaction_finished", "compaction_time_micros": 68208290, "compaction_time_cpu_micros": 28465039, "output_level": 3, "num_output_files": 19, "total_output_size": 691903557, "num_input_records": 16947370, "num_output_records": 16850971, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 599451242, "file_range_sync_nanos": 0, "file_fsync_nanos": 14702, "file_prepare_write_nanos": 57525, "lsm_state": [0, 0, 0, 19, 0, 0, 0]} 2024/11/23-20:18:20.125185 139957164807744 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732364231,"executesMs":68457,"compactionJobId":35016,"attempt":0,"compactionInputRawBytes":2449419652,"compactionInputZipBytes":712321378,"compactionOutputRawBytes":0,"compactionOutputZipBytes":691903557,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-8bd61eedd5e191310d5864250f6ecad5-f2ad0173c4d469b2-00"}}, response = 2024/11/23-20:18:20.125284 139957164807744 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-35016/att-00: olev 3, work 68.457 s, result 5.918 ms, install 1.546 ms, input{raw 2.281 GiB zip 679.323 MiB}