2024/11/22-18:43:57.969975 140024961644096 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00: fork to child time = 0.009626 sec 2024/11/22-18:43:57.970869 140024961644096 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00 2024/11/22-18:43:57.978114 140024961644096 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00 2024/11/22-18:43:57.979233 140024961644096 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-18:43:57.979489 140024961644096 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-18:43:58.029691 140024961644096 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-07849/att-00/MANIFEST-206347 2024/11/22-18:43:58.062520 140024961644096 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-07849/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 221181, last_sequence is 3474972017, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 221045 2024/11/22-18:43:58.062528 140024961644096 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-18:43:58.062623 140024961644096 [version_set.cc:5721] Creating manifest 221181 2024/11/22-18:43:58.098983 140024961644096 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-18:43:58.120373 140024961644096 [compaction_job.cc:2437] [default] [JOB 7849] Compacting 1@2 + 15@3 files to L3, score 0.96, subcompactions 1 : 1 2024/11/22-18:43:58.120397 140024961644096 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 5964 Base level 2, inputs: [221178(27MB)], [221116(35MB) 221117(34MB) 221118(36MB) 221119(35MB) 221120(33MB) 221121(35MB) 221122(35MB) 221123(34MB) 221124(35MB) 221132(35MB) 221133(36MB) 221134(35MB) 221135(35MB) 221136(36MB) 221137(35MB)] 2024/11/22-18:43:58.128003 140024961644096 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-18:43:58.128036 140024961644096 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-18:43:58.128058 140024961644096 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-18:43:58.128063 140024961644096 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-18:43:58.128074 140024961644096 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-18:43:58.583686 140024961644096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221182.sst) /dev/shm/ToplingTemp/Topling-244999-mQWaLQ 2024/11/22-18:43:59.367411 140024961644096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221182.sst): first pass time = 0.78's, 171.266'MB/sec 2024/11/22-18:44:01.081430 140024961644096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221182.sst): old prealloc_size = 147639500, real_size = 34045792 2024/11/22-18:44:01.082031 140024961644096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221182.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926262 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0321 GB avg-val = 116.90 avg-zval = 34.60 2024/11/22-18:44:01.082639 140024961644096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221182.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:44:01.950353 140024961644096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221182.sst): second pass time = 1.71's, 63.201'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.873'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3041.333'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.315'MB/sec write SST data time = 0.13's, 278.457'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 37597.731'MB/sec zip my value time = 1.71's, unzip length = 0.108'GB zip my value throughput = 63.201'MB/sec zip pipeline throughput = 63.201'MB/sec entries = 926262 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.90 avg-zval = 34.60 usrkeys = 926262 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.84 avg-zval = 34.60 TagRS{ kinds = RS_Key0_TagN bytes = 144840 } TagArray{ size = 653157 bytes = 2204416 } seq expand size = 4572099 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0321 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5142 value = 3.3783 dict = 1.00 all = 3.6029 } Zip/UnZip{ index = 0.0951 value = 0.2960 dict = 1.00 all = 0.2776 } ---------------------------- total value len = 0.112855 GB avg = 0.122 KB (by entry num) total key len = 0.025935 GB avg = 0.028 KB total ukey len = 0.018525 GB avg = 0.020 KB total ukey num = 0.000926262 Billion total entry num = 0.000926262 Billion write speed all = 0.003909700 MB/sec (with seq num) write speed all = 0.003700958 MB/sec (without seq num) 2024/11/22-18:44:01.982671 140024961644096 [compaction_job.cc:1948] [default] [JOB 7849] Generated table #221182: 926262 keys, 37253074 bytes, temperature: kWarm 2024/11/22-18:44:01.982770 140024961644096 EVENT_LOG_v1 {"time_micros": 1732272241982716, "cf_name": "default", "job": 7849, "event": "table_file_creation", "file_number": 221182, "file_size": 37253074, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3474945647, "table_properties": {"data_size": 32052112, "index_size": 1761920, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25935336, "raw_average_key_size": 28, "raw_value_size": 108282424, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926262, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272238, "oldest_key_time": 0, "file_creation_time": 1732272238, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221182, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:44:02.022691 140024961644096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221183.sst) /dev/shm/ToplingTemp/Topling-244999-W21zdU 2024/11/22-18:44:03.247620 140024961644096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221183.sst): first pass time = 1.22's, 109.576'MB/sec 2024/11/22-18:44:04.989103 140024961644096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221183.sst): old prealloc_size = 147639500, real_size = 34976560 2024/11/22-18:44:04.989788 140024961644096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221183.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1081638 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1039 GB zip-val = 0.0326 GB avg-val = 96.09 avg-zval = 30.18 2024/11/22-18:44:04.989829 140024961644096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221183.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:44:05.860800 140024961644096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221183.sst): second pass time = 1.74's, 59.713'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 41.017'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3096.151'MB/sec (index lex order gen) rebuild zvType time = 0.61's, 0.441'MB/sec write SST data time = 0.26's, 138.953'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.035'MB, throughput = 34639.308'MB/sec zip my value time = 1.74's, unzip length = 0.104'GB zip my value throughput = 59.713'MB/sec zip pipeline throughput = 59.713'MB/sec entries = 1081638 avg-key = 28.00 avg-zkey = 1.90 avg-val = 96.09 avg-zval = 30.18 usrkeys = 1081638 avg-key = 20.00 avg-zkey = 1.90 avg-val = 96.09 avg-zval = 30.18 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0216 GB value = 0.1039 GB dict = 1.04 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0326 GB dict = 1.04 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5165 value = 3.1833 dict = 1.00 all = 3.7541 } Zip/UnZip{ index = 0.0951 value = 0.3141 dict = 1.00 all = 0.2664 } ---------------------------- total value len = 0.216786 GB avg = 0.108 KB (by entry num) total key len = 0.056221 GB avg = 0.028 KB total ukey len = 0.040158 GB avg = 0.020 KB total ukey num = 0.002007900 Billion total entry num = 0.002007900 Billion write speed all = 0.007689758 MB/sec (with seq num) write speed all = 0.007237308 MB/sec (without seq num) 2024/11/22-18:44:05.896918 140024961644096 [compaction_job.cc:1948] [default] [JOB 7849] Generated table #221183: 1081638 keys, 35752259 bytes, temperature: kWarm 2024/11/22-18:44:05.896999 140024961644096 EVENT_LOG_v1 {"time_micros": 1732272245896956, "cf_name": "default", "job": 7849, "event": "table_file_creation", "file_number": 221183, "file_size": 35752259, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 32648928, "index_size": 2057024, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30285864, "raw_average_key_size": 28, "raw_value_size": 103931933, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1081638, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272242, "oldest_key_time": 0, "file_creation_time": 1732272242, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221183, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:44:05.935023 140024961644096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221184.sst) /dev/shm/ToplingTemp/Topling-244999-J2h5KQ 2024/11/22-18:44:07.112255 140024961644096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221184.sst): first pass time = 1.18's, 114.016'MB/sec 2024/11/22-18:44:10.095555 140024961644096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221184.sst): old prealloc_size = 147639500, real_size = 33796544 2024/11/22-18:44:10.096262 140024961644096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221184.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922992 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0318 GB avg-val = 117.42 avg-zval = 34.45 2024/11/22-18:44:10.097085 140024961644096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221184.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:44:11.417991 140024961644096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221184.sst): second pass time = 2.98's, 36.337'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.32's, 28.671'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2639.248'MB/sec (index lex order gen) rebuild zvType time = 1.08's, 0.213'MB/sec write SST data time = 0.24's, 157.408'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 31265.681'MB/sec zip my value time = 2.98's, unzip length = 0.108'GB zip my value throughput = 36.337'MB/sec zip pipeline throughput = 100.175'MB/sec entries = 922992 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.42 avg-zval = 34.45 usrkeys = 922992 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.16 avg-zval = 34.45 TagRS{ kinds = RS_Key0_TagN bytes = 144320 } TagArray{ size = 889332 bytes = 3112672 } seq expand size = 6225324 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.4437 value = 3.4082 dict = 1.00 all = 3.5400 } Zip/UnZip{ index = 0.0958 value = 0.2934 dict = 1.00 all = 0.2825 } ---------------------------- total value len = 0.331386 GB avg = 0.113 KB (by entry num) total key len = 0.082065 GB avg = 0.028 KB total ukey len = 0.058618 GB avg = 0.020 KB total ukey num = 0.002930892 Billion total entry num = 0.002930892 Billion write speed all = 0.011643771 MB/sec (with seq num) write speed all = 0.010983443 MB/sec (without seq num) 2024/11/22-18:44:11.453360 140024961644096 [compaction_job.cc:1948] [default] [JOB 7849] Generated table #221184: 922992 keys, 37914876 bytes, temperature: kWarm 2024/11/22-18:44:11.453422 140024961644096 EVENT_LOG_v1 {"time_micros": 1732272251453390, "cf_name": "default", "job": 7849, "event": "table_file_creation", "file_number": 221184, "file_size": 37914876, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3474946033, "table_properties": {"data_size": 31798064, "index_size": 1767552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25843776, "raw_average_key_size": 28, "raw_value_size": 108373996, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922992, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272245, "oldest_key_time": 0, "file_creation_time": 1732272245, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221184, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:44:11.487935 140024961644096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221185.sst) /dev/shm/ToplingTemp/Topling-244999-YwnEDQ 2024/11/22-18:44:12.895855 140024961644096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221185.sst): first pass time = 1.41's, 95.333'MB/sec 2024/11/22-18:44:15.869540 140024961644096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221185.sst): old prealloc_size = 147639500, real_size = 33192704 2024/11/22-18:44:15.870073 140024961644096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221185.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925085 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 117.09 avg-zval = 33.72 2024/11/22-18:44:15.870833 140024961644096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221185.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:44:17.235939 140024961644096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221185.sst): second pass time = 2.97's, 36.433'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.37's, 27.204'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3509.851'MB/sec (index lex order gen) rebuild zvType time = 1.02's, 0.226'MB/sec write SST data time = 0.34's, 107.823'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 28575.371'MB/sec zip my value time = 2.97's, unzip length = 0.108'GB zip my value throughput = 36.433'MB/sec zip pipeline throughput = 110.341'MB/sec entries = 925085 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.09 avg-zval = 33.72 usrkeys = 925085 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.09 avg-zval = 33.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925085 bytes = 3122176 } seq expand size = 6475595 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.4440 value = 3.4728 dict = 1.00 all = 3.6109 } Zip/UnZip{ index = 0.0957 value = 0.2880 dict = 1.00 all = 0.2769 } ---------------------------- total value len = 0.446177 GB avg = 0.116 KB (by entry num) total key len = 0.107967 GB avg = 0.028 KB total ukey len = 0.077120 GB avg = 0.020 KB total ukey num = 0.003855977 Billion total entry num = 0.003855977 Billion write speed all = 0.015603479 MB/sec (with seq num) write speed all = 0.014734872 MB/sec (without seq num) 2024/11/22-18:44:17.272807 140024961644096 [compaction_job.cc:1948] [default] [JOB 7849] Generated table #221185: 925085 keys, 37170580 bytes, temperature: kWarm 2024/11/22-18:44:17.272892 140024961644096 EVENT_LOG_v1 {"time_micros": 1732272257272845, "cf_name": "default", "job": 7849, "event": "table_file_creation", "file_number": 221185, "file_size": 37170580, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3365258144, "largest_seqno": 3474946065, "table_properties": {"data_size": 31189744, "index_size": 1771520, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25902380, "raw_average_key_size": 28, "raw_value_size": 108315410, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925085, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272251, "oldest_key_time": 0, "file_creation_time": 1732272251, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221185, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:44:17.321563 140024961644096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221186.sst) /dev/shm/ToplingTemp/Topling-244999-7JJgDS 2024/11/22-18:44:18.637981 140024961644096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221186.sst): first pass time = 1.32's, 101.960'MB/sec 2024/11/22-18:44:20.261265 140024961644096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221186.sst): old prealloc_size = 147639500, real_size = 32035776 2024/11/22-18:44:20.261834 140024961644096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221186.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920509 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0301 GB avg-val = 117.81 avg-zval = 32.65 2024/11/22-18:44:20.262598 140024961644096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221186.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:44:21.282170 140024961644096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221186.sst): second pass time = 1.62's, 66.831'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.02's, 35.379'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3213.734'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.294'MB/sec write SST data time = 0.24's, 151.325'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 28984.550'MB/sec zip my value time = 1.62's, unzip length = 0.108'GB zip my value throughput = 66.831'MB/sec zip pipeline throughput = 124.242'MB/sec entries = 920509 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.81 avg-zval = 32.65 usrkeys = 920509 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.81 avg-zval = 32.65 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 920509 bytes = 3221792 } seq expand size = 6443563 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.5112 value = 3.6083 dict = 1.00 all = 3.7161 } Zip/UnZip{ index = 0.0951 value = 0.2771 dict = 1.00 all = 0.2691 } ---------------------------- total value len = 0.561064 GB avg = 0.117 KB (by entry num) total key len = 0.133742 GB avg = 0.028 KB total ukey len = 0.095530 GB avg = 0.020 KB total ukey num = 0.004776486 Billion total entry num = 0.004776486 Billion write speed all = 0.019561963 MB/sec (with seq num) write speed all = 0.018486123 MB/sec (without seq num) 2024/11/22-18:44:21.318570 140024961644096 [compaction_job.cc:1948] [default] [JOB 7849] Generated table #221186: 920509 keys, 36117608 bytes, temperature: kWarm 2024/11/22-18:44:21.318650 140024961644096 EVENT_LOG_v1 {"time_micros": 1732272261318606, "cf_name": "default", "job": 7849, "event": "table_file_creation", "file_number": 221186, "file_size": 36117608, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3474944326, "table_properties": {"data_size": 30054000, "index_size": 1751488, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25774252, "raw_average_key_size": 28, "raw_value_size": 108443498, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920509, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272257, "oldest_key_time": 0, "file_creation_time": 1732272257, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221186, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:44:21.395021 140024961644096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221187.sst) /dev/shm/ToplingTemp/Topling-244999-g4BqnQ 2024/11/22-18:44:22.593335 140024961644096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221187.sst): first pass time = 1.20's, 112.011'MB/sec 2024/11/22-18:44:24.074258 140024961644096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221187.sst): old prealloc_size = 147639500, real_size = 33647888 2024/11/22-18:44:24.074872 140024961644096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221187.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923092 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0316 GB avg-val = 117.40 avg-zval = 34.29 2024/11/22-18:44:24.075652 140024961644096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221187.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:44:25.361521 140024961644096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221187.sst): second pass time = 1.48's, 73.213'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.29's, 29.316'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2993.309'MB/sec (index lex order gen) rebuild zvType time = 1.03's, 0.223'MB/sec write SST data time = 0.25's, 149.382'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32330.607'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.213'MB/sec zip pipeline throughput = 73.213'MB/sec entries = 923092 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.40 avg-zval = 34.29 usrkeys = 923092 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.40 avg-zval = 34.29 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923092 bytes = 3230832 } seq expand size = 6461644 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.4430 value = 3.4242 dict = 1.00 all = 3.5567 } Zip/UnZip{ index = 0.0958 value = 0.2920 dict = 1.00 all = 0.2812 } ---------------------------- total value len = 0.675897 GB avg = 0.119 KB (by entry num) total key len = 0.159588 GB avg = 0.028 KB total ukey len = 0.113992 GB avg = 0.020 KB total ukey num = 0.005699578 Billion total entry num = 0.005699578 Billion write speed all = 0.023520036 MB/sec (with seq num) write speed all = 0.022236429 MB/sec (without seq num) 2024/11/22-18:44:25.403481 140024961644096 [compaction_job.cc:1948] [default] [JOB 7849] Generated table #221187: 923092 keys, 37736722 bytes, temperature: kWarm 2024/11/22-18:44:25.403559 140024961644096 EVENT_LOG_v1 {"time_micros": 1732272265403515, "cf_name": "default", "job": 7849, "event": "table_file_creation", "file_number": 221187, "file_size": 37736722, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701122, "largest_seqno": 3474946091, "table_properties": {"data_size": 31649088, "index_size": 1767872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25846576, "raw_average_key_size": 28, "raw_value_size": 108371280, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923092, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272261, "oldest_key_time": 0, "file_creation_time": 1732272261, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221187, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:44:25.468913 140024961644096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221188.sst) /dev/shm/ToplingTemp/Topling-244999-MDXn4Q 2024/11/22-18:44:26.423858 140024961644096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221188.sst): first pass time = 0.95's, 140.558'MB/sec 2024/11/22-18:44:28.258604 140024961644096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221188.sst): old prealloc_size = 147639500, real_size = 33532176 2024/11/22-18:44:28.259178 140024961644096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221188.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926039 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 116.94 avg-zval = 34.06 2024/11/22-18:44:28.259938 140024961644096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221188.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:44:29.168796 140024961644096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221188.sst): second pass time = 1.83's, 59.042'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 41.342'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3194.760'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.296'MB/sec write SST data time = 0.13's, 294.255'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 27118.713'MB/sec zip my value time = 1.83's, unzip length = 0.108'GB zip my value throughput = 59.042'MB/sec zip pipeline throughput = 97.821'MB/sec entries = 926039 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.94 avg-zval = 34.06 usrkeys = 926039 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.94 avg-zval = 34.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926039 bytes = 3241152 } seq expand size = 6482273 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5136 value = 3.4335 dict = 1.00 all = 3.5669 } Zip/UnZip{ index = 0.0951 value = 0.2912 dict = 1.00 all = 0.2804 } ---------------------------- total value len = 0.790668 GB avg = 0.119 KB (by entry num) total key len = 0.185517 GB avg = 0.028 KB total ukey len = 0.132512 GB avg = 0.020 KB total ukey num = 0.006625617 Billion total entry num = 0.006625617 Billion write speed all = 0.027477988 MB/sec (with seq num) write speed all = 0.025985987 MB/sec (without seq num) 2024/11/22-18:44:29.204179 140024961644096 [compaction_job.cc:1948] [default] [JOB 7849] Generated table #221188: 926039 keys, 37628182 bytes, temperature: kWarm 2024/11/22-18:44:29.204239 140024961644096 EVENT_LOG_v1 {"time_micros": 1732272269204206, "cf_name": "default", "job": 7849, "event": "table_file_creation", "file_number": 221188, "file_size": 37628182, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3326330223, "largest_seqno": 3474946110, "table_properties": {"data_size": 31538880, "index_size": 1761600, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25929092, "raw_average_key_size": 28, "raw_value_size": 108288682, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926039, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272265, "oldest_key_time": 0, "file_creation_time": 1732272265, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221188, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:44:29.242088 140024961644096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221189.sst) /dev/shm/ToplingTemp/Topling-244999-QurzEU 2024/11/22-18:44:30.081571 140024961644096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221189.sst): first pass time = 0.84's, 159.889'MB/sec 2024/11/22-18:44:31.635808 140024961644096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221189.sst): old prealloc_size = 147639500, real_size = 32733184 2024/11/22-18:44:31.636360 140024961644096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221189.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922780 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0307 GB avg-val = 117.45 avg-zval = 33.32 2024/11/22-18:44:31.636920 140024961644096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221189.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:44:32.528411 140024961644096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221189.sst): second pass time = 1.55's, 69.795'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 40.188'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3312.520'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.302'MB/sec write SST data time = 0.13's, 280.786'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 31017.669'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 69.795'MB/sec zip pipeline throughput = 114.499'MB/sec entries = 922780 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.45 avg-zval = 33.32 usrkeys = 922780 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.08 avg-zval = 33.32 TagRS{ kinds = RS_Key0_TagN bytes = 144280 } TagArray{ size = 610305 bytes = 2136080 } seq expand size = 4272135 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0359 GB } UnZip/Zip{ index = 10.5106 value = 3.5250 dict = 1.00 all = 3.7416 } Zip/UnZip{ index = 0.0951 value = 0.2837 dict = 1.00 all = 0.2673 } ---------------------------- total value len = 0.903320 GB avg = 0.120 KB (by entry num) total key len = 0.211355 GB avg = 0.028 KB total ukey len = 0.150968 GB avg = 0.020 KB total ukey num = 0.007548397 Billion total entry num = 0.007548397 Billion write speed all = 0.031373282 MB/sec (with seq num) write speed all = 0.029673644 MB/sec (without seq num) 2024/11/22-18:44:32.561476 140024961644096 [compaction_job.cc:1948] [default] [JOB 7849] Generated table #221189: 922780 keys, 35871794 bytes, temperature: kWarm 2024/11/22-18:44:32.561540 140024961644096 EVENT_LOG_v1 {"time_micros": 1732272272561508, "cf_name": "default", "job": 7849, "event": "table_file_creation", "file_number": 221189, "file_size": 35871794, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3474946127, "table_properties": {"data_size": 30746416, "index_size": 1755904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25837840, "raw_average_key_size": 28, "raw_value_size": 108379916, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922780, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272269, "oldest_key_time": 0, "file_creation_time": 1732272269, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221189, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:44:32.598125 140024961644096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221190.sst) /dev/shm/ToplingTemp/Topling-244999-mb7UhR 2024/11/22-18:44:33.500495 140024961644096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221190.sst): first pass time = 0.90's, 148.747'MB/sec 2024/11/22-18:44:34.916030 140024961644096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221190.sst): old prealloc_size = 147639500, real_size = 33607584 2024/11/22-18:44:34.916582 140024961644096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221190.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923447 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0316 GB avg-val = 117.34 avg-zval = 34.24 2024/11/22-18:44:34.917360 140024961644096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221190.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:44:35.849522 140024961644096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221190.sst): second pass time = 1.41's, 76.589'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.260'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3319.660'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.287'MB/sec write SST data time = 0.13's, 295.140'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34904.022'MB/sec zip my value time = 1.41's, unzip length = 0.108'GB zip my value throughput = 76.589'MB/sec zip pipeline throughput = 76.589'MB/sec entries = 923447 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.34 avg-zval = 34.24 usrkeys = 923447 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.34 avg-zval = 34.24 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923447 bytes = 3116656 } seq expand size = 6464129 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5140 value = 3.4270 dict = 1.00 all = 3.5713 } Zip/UnZip{ index = 0.0951 value = 0.2918 dict = 1.00 all = 0.2800 } ---------------------------- total value len = 1.018145 GB avg = 0.120 KB (by entry num) total key len = 0.237212 GB avg = 0.028 KB total ukey len = 0.169437 GB avg = 0.020 KB total ukey num = 0.008471844 Billion total entry num = 0.008471844 Billion write speed all = 0.035329568 MB/sec (with seq num) write speed all = 0.033422180 MB/sec (without seq num) 2024/11/22-18:44:35.884217 140024961644096 [compaction_job.cc:1948] [default] [JOB 7849] Generated table #221190: 923447 keys, 37582154 bytes, temperature: kWarm 2024/11/22-18:44:35.884282 140024961644096 EVENT_LOG_v1 {"time_micros": 1732272275884248, "cf_name": "default", "job": 7849, "event": "table_file_creation", "file_number": 221190, "file_size": 37582154, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3388810268, "largest_seqno": 3474946163, "table_properties": {"data_size": 31619920, "index_size": 1756608, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25856516, "raw_average_key_size": 28, "raw_value_size": 108361302, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923447, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272272, "oldest_key_time": 0, "file_creation_time": 1732272272, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221190, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:44:35.918935 140024961644096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221191.sst) /dev/shm/ToplingTemp/Topling-244999-pHi7eQ 2024/11/22-18:44:36.733409 140024961644096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221191.sst): first pass time = 0.81's, 164.798'MB/sec 2024/11/22-18:44:38.182497 140024961644096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221191.sst): old prealloc_size = 147639500, real_size = 33302480 2024/11/22-18:44:38.183041 140024961644096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221191.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927113 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.77 avg-zval = 33.77 2024/11/22-18:44:38.183734 140024961644096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221191.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:44:39.077848 140024961644096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221191.sst): second pass time = 1.45's, 74.744'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 41.298'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3367.932'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.303'MB/sec write SST data time = 0.13's, 283.947'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 30612.029'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.744'MB/sec zip pipeline throughput = 83.465'MB/sec entries = 927113 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.77 avg-zval = 33.77 usrkeys = 927113 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.74 avg-zval = 33.77 TagRS{ kinds = RS_Key0_TagN bytes = 144960 } TagArray{ size = 791092 bytes = 2669952 } seq expand size = 5537644 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5098 value = 3.4581 dict = 1.00 all = 3.6299 } Zip/UnZip{ index = 0.0951 value = 0.2892 dict = 1.00 all = 0.2755 } ---------------------------- total value len = 1.131941 GB avg = 0.120 KB (by entry num) total key len = 0.263171 GB avg = 0.028 KB total ukey len = 0.187979 GB avg = 0.020 KB total ukey num = 0.009398957 Billion total entry num = 0.009398957 Billion write speed all = 0.039259144 MB/sec (with seq num) write speed all = 0.037143214 MB/sec (without seq num) 2024/11/22-18:44:39.113842 140024961644096 [compaction_job.cc:1948] [default] [JOB 7849] Generated table #221191: 927113 keys, 36975758 bytes, temperature: kWarm 2024/11/22-18:44:39.113903 140024961644096 EVENT_LOG_v1 {"time_micros": 1732272279113871, "cf_name": "default", "job": 7849, "event": "table_file_creation", "file_number": 221191, "file_size": 36975758, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3474946267, "table_properties": {"data_size": 31306240, "index_size": 1764288, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25959164, "raw_average_key_size": 28, "raw_value_size": 108258586, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927113, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272275, "oldest_key_time": 0, "file_creation_time": 1732272275, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221191, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:44:39.152557 140024961644096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221192.sst) /dev/shm/ToplingTemp/Topling-244999-mhaPhQ 2024/11/22-18:44:39.983752 140024961644096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221192.sst): first pass time = 0.83's, 161.483'MB/sec 2024/11/22-18:44:41.543052 140024961644096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221192.sst): old prealloc_size = 147639500, real_size = 36259888 2024/11/22-18:44:41.543591 140024961644096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221192.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923873 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0343 GB avg-val = 117.28 avg-zval = 37.10 2024/11/22-18:44:41.543978 140024961644096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221192.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:44:42.294888 140024961644096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221192.sst): second pass time = 1.56's, 69.513'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.75's, 51.207'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3391.470'MB/sec (index lex order gen) rebuild zvType time = 0.62's, 0.370'MB/sec write SST data time = 0.13's, 303.437'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 28760.525'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.513'MB/sec zip pipeline throughput = 69.513'MB/sec entries = 923873 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.28 avg-zval = 37.10 usrkeys = 923873 avg-key = 20.00 avg-zkey = 1.90 avg-val = 120.05 avg-zval = 37.10 TagRS{ kinds = RS_Key0_TagN bytes = 144440 } TagArray{ size = 365595 bytes = 1233904 } seq expand size = 2559165 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0343 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.5135 value = 3.1615 dict = 1.00 all = 3.4863 } Zip/UnZip{ index = 0.0951 value = 0.3163 dict = 1.00 all = 0.2868 } ---------------------------- total value len = 1.242850 GB avg = 0.120 KB (by entry num) total key len = 0.289039 GB avg = 0.028 KB total ukey len = 0.206457 GB avg = 0.020 KB total ukey num = 0.010322830 Billion total entry num = 0.010322830 Billion write speed all = 0.043104215 MB/sec (with seq num) write speed all = 0.040780509 MB/sec (without seq num) 2024/11/22-18:44:42.328294 140024961644096 [compaction_job.cc:1948] [default] [JOB 7849] Generated table #221192: 923873 keys, 38498805 bytes, temperature: kWarm 2024/11/22-18:44:42.328356 140024961644096 EVENT_LOG_v1 {"time_micros": 1732272282328323, "cf_name": "default", "job": 7849, "event": "table_file_creation", "file_number": 221192, "file_size": 38498805, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3343247035, "table_properties": {"data_size": 34271200, "index_size": 1757504, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25868444, "raw_average_key_size": 28, "raw_value_size": 108349414, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923873, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272279, "oldest_key_time": 0, "file_creation_time": 1732272279, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221192, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:44:42.363123 140024961644096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221193.sst) /dev/shm/ToplingTemp/Topling-244999-WUbJNR 2024/11/22-18:44:43.176310 140024961644096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221193.sst): first pass time = 0.81's, 165.060'MB/sec 2024/11/22-18:44:44.767646 140024961644096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221193.sst): old prealloc_size = 147639500, real_size = 33494464 2024/11/22-18:44:44.768204 140024961644096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221193.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922788 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0315 GB avg-val = 117.45 avg-zval = 34.14 2024/11/22-18:44:44.768954 140024961644096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221193.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:44:45.685924 140024961644096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221193.sst): second pass time = 1.59's, 68.134'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.802'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3297.182'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.292'MB/sec write SST data time = 0.13's, 295.287'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 30166.082'MB/sec zip my value time = 1.59's, unzip length = 0.108'GB zip my value throughput = 68.134'MB/sec zip pipeline throughput = 125.343'MB/sec entries = 922788 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.45 avg-zval = 34.14 usrkeys = 922788 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.45 avg-zval = 34.14 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922788 bytes = 3114432 } seq expand size = 6459516 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5134 value = 3.4397 dict = 1.00 all = 3.5823 } Zip/UnZip{ index = 0.0951 value = 0.2907 dict = 1.00 all = 0.2792 } ---------------------------- total value len = 1.357689 GB avg = 0.121 KB (by entry num) total key len = 0.314877 GB avg = 0.028 KB total ukey len = 0.224912 GB avg = 0.020 KB total ukey num = 0.011245618 Billion total entry num = 0.011245618 Billion write speed all = 0.047058097 MB/sec (with seq num) write speed all = 0.044526910 MB/sec (without seq num) 2024/11/22-18:44:45.718153 140024961644096 [compaction_job.cc:1948] [default] [JOB 7849] Generated table #221193: 922788 keys, 37466982 bytes, temperature: kWarm 2024/11/22-18:44:45.718211 140024961644096 EVENT_LOG_v1 {"time_micros": 1732272285718181, "cf_name": "default", "job": 7849, "event": "table_file_creation", "file_number": 221193, "file_size": 37466982, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3343247827, "largest_seqno": 3474945825, "table_properties": {"data_size": 31508144, "index_size": 1755456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25838064, "raw_average_key_size": 28, "raw_value_size": 108379800, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922788, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272282, "oldest_key_time": 0, "file_creation_time": 1732272282, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221193, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:44:45.762158 140024961644096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221194.sst) /dev/shm/ToplingTemp/Topling-244999-EGgFcR 2024/11/22-18:44:46.639557 140024961644096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221194.sst): first pass time = 0.88's, 152.980'MB/sec 2024/11/22-18:44:47.969914 140024961644096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221194.sst): old prealloc_size = 147639500, real_size = 33364016 2024/11/22-18:44:47.970478 140024961644096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221194.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922672 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0314 GB avg-val = 117.47 avg-zval = 34.01 2024/11/22-18:44:47.971244 140024961644096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221194.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:44:48.893349 140024961644096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221194.sst): second pass time = 1.33's, 81.543'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.557'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3224.061'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.290'MB/sec write SST data time = 0.13's, 293.880'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33785.861'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.543'MB/sec zip pipeline throughput = 81.543'MB/sec entries = 922672 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.47 avg-zval = 34.01 usrkeys = 922672 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.47 avg-zval = 34.01 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922672 bytes = 3229360 } seq expand size = 6458704 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5120 value = 3.4541 dict = 1.00 all = 3.5838 } Zip/UnZip{ index = 0.0951 value = 0.2895 dict = 1.00 all = 0.2790 } ---------------------------- total value len = 1.472531 GB avg = 0.121 KB (by entry num) total key len = 0.340712 GB avg = 0.028 KB total ukey len = 0.243366 GB avg = 0.020 KB total ukey num = 0.012168290 Billion total entry num = 0.012168290 Billion write speed all = 0.051011463 MB/sec (with seq num) write speed all = 0.048272846 MB/sec (without seq num) 2024/11/22-18:44:48.930248 140024961644096 [compaction_job.cc:1948] [default] [JOB 7849] Generated table #221194: 922672 keys, 37451500 bytes, temperature: kWarm 2024/11/22-18:44:48.930315 140024961644096 EVENT_LOG_v1 {"time_micros": 1732272288930277, "cf_name": "default", "job": 7849, "event": "table_file_creation", "file_number": 221194, "file_size": 37451500, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701530, "largest_seqno": 3474945984, "table_properties": {"data_size": 31377696, "index_size": 1755456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25834816, "raw_average_key_size": 28, "raw_value_size": 108382968, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922672, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272285, "oldest_key_time": 0, "file_creation_time": 1732272285, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221194, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:44:48.982465 140024961644096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221195.sst) /dev/shm/ToplingTemp/Topling-244999-jpZVwU 2024/11/22-18:44:49.950366 140024961644096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221195.sst): first pass time = 0.97's, 138.675'MB/sec 2024/11/22-18:44:51.766454 140024961644096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221195.sst): old prealloc_size = 147639500, real_size = 33814448 2024/11/22-18:44:51.767017 140024961644096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221195.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922865 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0318 GB avg-val = 117.44 avg-zval = 34.49 2024/11/22-18:44:51.767770 140024961644096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221195.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:44:52.880305 140024961644096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221195.sst): second pass time = 1.81's, 59.714'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.11's, 34.029'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3249.003'MB/sec (index lex order gen) rebuild zvType time = 0.88's, 0.262'MB/sec write SST data time = 0.23's, 163.474'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 27001.496'MB/sec zip my value time = 1.81's, unzip length = 0.108'GB zip my value throughput = 59.714'MB/sec zip pipeline throughput = 132.239'MB/sec entries = 922865 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.44 avg-zval = 34.49 usrkeys = 922865 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.44 avg-zval = 34.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922865 bytes = 3230048 } seq expand size = 6460055 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.5142 value = 3.4051 dict = 1.00 all = 3.5411 } Zip/UnZip{ index = 0.0951 value = 0.2937 dict = 1.00 all = 0.2824 } ---------------------------- total value len = 1.587369 GB avg = 0.121 KB (by entry num) total key len = 0.366552 GB avg = 0.028 KB total ukey len = 0.261823 GB avg = 0.020 KB total ukey num = 0.013091155 Billion total entry num = 0.013091155 Billion write speed all = 0.054962948 MB/sec (with seq num) write speed all = 0.052016960 MB/sec (without seq num) 2024/11/22-18:44:52.928758 140024961644096 [compaction_job.cc:1948] [default] [JOB 7849] Generated table #221195: 922865 keys, 37902684 bytes, temperature: kWarm 2024/11/22-18:44:52.928845 140024961644096 EVENT_LOG_v1 {"time_micros": 1732272292928801, "cf_name": "default", "job": 7849, "event": "table_file_creation", "file_number": 221195, "file_size": 37902684, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701550, "largest_seqno": 3474946226, "table_properties": {"data_size": 31828064, "index_size": 1755456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25840220, "raw_average_key_size": 28, "raw_value_size": 108377562, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922865, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272288, "oldest_key_time": 0, "file_creation_time": 1732272288, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221195, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:44:52.977416 140024961644096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221196.sst) /dev/shm/ToplingTemp/Topling-244999-o6utBT 2024/11/22-18:44:53.818093 140024961644096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221196.sst): first pass time = 0.84's, 159.664'MB/sec 2024/11/22-18:44:55.137068 140024961644096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221196.sst): old prealloc_size = 147639500, real_size = 33587008 2024/11/22-18:44:55.137597 140024961644096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221196.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922899 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0316 GB avg-val = 117.43 avg-zval = 34.24 2024/11/22-18:44:55.138356 140024961644096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221196.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:44:56.067051 140024961644096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221196.sst): second pass time = 1.32's, 82.208'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.389'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3484.428'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.288'MB/sec write SST data time = 0.13's, 293.140'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32233.180'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.208'MB/sec zip pipeline throughput = 82.208'MB/sec entries = 922899 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.43 avg-zval = 34.24 usrkeys = 922899 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.43 avg-zval = 34.24 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922899 bytes = 3114800 } seq expand size = 6460293 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5139 value = 3.4296 dict = 1.00 all = 3.5734 } Zip/UnZip{ index = 0.0951 value = 0.2916 dict = 1.00 all = 0.2798 } ---------------------------- total value len = 1.702206 GB avg = 0.121 KB (by entry num) total key len = 0.392394 GB avg = 0.028 KB total ukey len = 0.280281 GB avg = 0.020 KB total ukey num = 0.014014054 Billion total entry num = 0.014014054 Billion write speed all = 0.058914881 MB/sec (with seq num) write speed all = 0.055761490 MB/sec (without seq num) 2024/11/22-18:44:56.102355 140024961644096 [compaction_job.cc:1948] [default] [JOB 7849] Generated table #221196: 922899 keys, 37560076 bytes, temperature: kWarm 2024/11/22-18:44:56.102418 140024961644096 EVENT_LOG_v1 {"time_micros": 1732272296102386, "cf_name": "default", "job": 7849, "event": "table_file_creation", "file_number": 221196, "file_size": 37560076, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3354345306, "largest_seqno": 3474251816, "table_properties": {"data_size": 31600496, "index_size": 1755584, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25841172, "raw_average_key_size": 28, "raw_value_size": 108376634, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922899, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272292, "oldest_key_time": 0, "file_creation_time": 1732272292, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221196, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:44:56.142297 140024961644096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221197.sst) /dev/shm/ToplingTemp/Topling-244999-R7zdRS 2024/11/22-18:44:56.219229 140024961644096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221197.sst): first pass time = 0.08's, 233.185'MB/sec 2024/11/22-18:44:56.382550 140024961644096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221197.sst): old prealloc_size = 147639500, real_size = 5055824 2024/11/22-18:44:56.382678 140024961644096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221197.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 125927 prefix = 4 raw-key = 0.0020 GB zip-key = 0.0002 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0144 GB zip-val = 0.0048 GB avg-val = 114.37 avg-zval = 37.97 2024/11/22-18:44:56.382825 140024961644096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221197.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.018 GB 2024/11/22-18:44:56.504271 140024961644096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00/221197.sst): second pass time = 0.16's, 88.511'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.12's, 46.031'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2358.291'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.303'MB/sec write SST data time = 0.02's, 316.733'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.151'MB, throughput = 5484.135'MB/sec zip my value time = 0.16's, unzip length = 0.014'GB zip my value throughput = 88.511'MB/sec zip pipeline throughput = 88.511'MB/sec entries = 125927 avg-key = 28.00 avg-zkey = 1.93 avg-val = 114.37 avg-zval = 37.97 usrkeys = 125927 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.37 avg-zval = 37.97 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 125927 bytes = 425024 } seq expand size = 881489 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0025 GB value = 0.0144 GB dict = 0.15 MB all = 0.0179 GB } __ZipSize{ index = 0.0002 GB value = 0.0048 GB dict = 0.15 MB all = 0.0056 GB } UnZip/Zip{ index = 10.3859 value = 3.0121 dict = 1.00 all = 3.2002 } Zip/UnZip{ index = 0.0963 value = 0.3320 dict = 1.00 all = 0.3125 } ---------------------------- total value len = 1.717490 GB avg = 0.121 KB (by entry num) total key len = 0.395919 GB avg = 0.028 KB total ukey len = 0.282800 GB avg = 0.020 KB total ukey num = 0.014139981 Billion total entry num = 0.014139981 Billion write speed all = 0.059443229 MB/sec (with seq num) write speed all = 0.056261541 MB/sec (without seq num) 2024/11/22-18:44:56.514374 140024961644096 [compaction_job.cc:1948] [default] [JOB 7849] Generated table #221197: 125927 keys, 5602387 bytes, temperature: kWarm 2024/11/22-18:44:56.514434 140024961644096 EVENT_LOG_v1 {"time_micros": 1732272296514404, "cf_name": "default", "job": 7849, "event": "table_file_creation", "file_number": 221197, "file_size": 5602387, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268533, "largest_seqno": 3473439011, "table_properties": {"data_size": 4781632, "index_size": 242496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3525956, "raw_average_key_size": 28, "raw_value_size": 14402886, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 125927, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272296, "oldest_key_time": 0, "file_creation_time": 1732272296, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221197, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:44:56.526095 140024961644096 [compaction_job.cc:703] job-07849: subcompact[0], size: 0.564485 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07849/att-00 [221182,221183,221184,221185,221186,221187,221188,221189,221190,221191,221192,221193,221194,221195,221196,221197] 2024/11/22-18:44:56.644752 140024961644096 (Original Log Time 2024/11/22-18:43:58.120466) EVENT_LOG_v1 {"time_micros": 1732272238120412, "job": 7849, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [221178], "files_L3": [221116, 221117, 221118, 221119, 221120, 221121, 221122, 221123, 221124, 221132, 221133, 221134, 221135, 221136, 221137], "score": 0.955386, "input_data_size": 585208303, "oldest_snapshot_seqno": 3474971978} 2024/11/22-18:44:56.644755 140024961644096 (Original Log Time 2024/11/22-18:44:56.643959) [compaction_job.cc:2022] [default] [JOB 7849] Compacted 1@2 + 15@3 files to L3 => 564485441 bytes 2024/11/22-18:44:56.644756 140024961644096 (Original Log Time 2024/11/22-18:44:56.644684) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 16 0 0 0] max score 0.02, MB/sec: 10.0 rd, 9.7 wr, level 3, files in(1, 15) out(16 +0 blob) MB in(27.2, 530.9 +0.0 blob) out(538.3 +0.0 blob), read-write-amplify(40.3) write-amplify(19.8) OK, records in: 14250517, records dropped: 110536 output_compression: Snappy 2024/11/22-18:44:56.644762 140024961644096 (Original Log Time 2024/11/22-18:44:56.644731) EVENT_LOG_v1 {"time_micros": 1732272296644708, "cf": "default", "job": 7849, "event": "compaction_finished", "compaction_time_micros": 58405636, "compaction_time_cpu_micros": 25280826, "output_level": 3, "num_output_files": 16, "total_output_size": 564485441, "num_input_records": 14250517, "num_output_records": 14139981, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 411500984, "file_range_sync_nanos": 0, "file_fsync_nanos": 15905, "file_prepare_write_nanos": 40929, "lsm_state": [0, 0, 0, 16, 0, 0, 0]} 2024/11/22-18:44:56.779993 140024961644096 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732272237,"executesMs":58667,"compactionJobId":7849,"attempt":0,"compactionInputRawBytes":2046891871,"compactionInputZipBytes":585208303,"compactionOutputRawBytes":0,"compactionOutputZipBytes":564485441,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-69d3629f6b8d1c8eccd0dcde70e9782f-08c73300e7ffa631-00"}}, response = 2024/11/22-18:44:56.780083 140024961644096 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-07849/att-00: olev 3, work 58.667 s, result 6.288 ms, install 0.824 ms, input{raw 1.906 GiB zip 558.098 MiB}