2024/11/22-23:02:07.508998 140025205032512 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00: fork to child time = 0.009090 sec 2024/11/22-23:02:07.510072 140025205032512 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00 2024/11/22-23:02:07.517642 140025205032512 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00 2024/11/22-23:02:07.518740 140025205032512 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-23:02:07.519018 140025205032512 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-23:02:07.564663 140025205032512 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-13448/att-00/MANIFEST-206347 2024/11/22-23:02:07.594620 140025205032512 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-13448/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 232744, last_sequence is 3628634318, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 232597 2024/11/22-23:02:07.594628 140025205032512 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-23:02:07.594739 140025205032512 [version_set.cc:5721] Creating manifest 232744 2024/11/22-23:02:07.624392 140025205032512 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-23:02:07.642906 140025205032512 [compaction_job.cc:2437] [default] [JOB 13448] Compacting 1@2 + 15@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/22-23:02:07.642925 140025205032512 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 10105 Base level 2, inputs: [232734(27MB)], [232672(34MB) 232673(35MB) 232674(35MB) 232675(35MB) 232676(35MB) 232677(34MB) 232678(35MB) 232679(35MB) 232680(37MB) 232681(36MB) 232682(35MB) 232683(36MB) 232684(35MB) 232685(36MB) 232697(37MB)] 2024/11/22-23:02:07.648784 140025205032512 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-23:02:07.648816 140025205032512 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-23:02:07.648835 140025205032512 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-23:02:07.648841 140025205032512 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-23:02:07.648852 140025205032512 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-23:02:08.146312 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232745.sst) /dev/shm/ToplingTemp/Topling-325230-15wjFx 2024/11/22-23:02:08.974736 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232745.sst): first pass time = 0.83's, 162.027'MB/sec 2024/11/22-23:02:11.314462 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232745.sst): old prealloc_size = 147639500, real_size = 32649616 2024/11/22-23:02:11.315061 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232745.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925015 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0307 GB avg-val = 117.10 avg-zval = 33.14 2024/11/22-23:02:11.315860 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232745.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:02:12.251247 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232745.sst): second pass time = 2.34's, 46.310'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.099'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3041.792'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.287'MB/sec write SST data time = 0.13's, 282.033'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 26457.302'MB/sec zip my value time = 2.34's, unzip length = 0.108'GB zip my value throughput = 46.310'MB/sec zip pipeline throughput = 65.041'MB/sec entries = 925015 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.10 avg-zval = 33.14 usrkeys = 925015 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.10 avg-zval = 33.14 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925015 bytes = 3121936 } seq expand size = 6475105 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5146 value = 3.5330 dict = 1.00 all = 3.6644 } Zip/UnZip{ index = 0.0951 value = 0.2830 dict = 1.00 all = 0.2729 } ---------------------------- total value len = 0.114793 GB avg = 0.124 KB (by entry num) total key len = 0.025900 GB avg = 0.028 KB total ukey len = 0.018500 GB avg = 0.020 KB total ukey num = 0.000925015 Billion total entry num = 0.000925015 Billion write speed all = 0.002759272 MB/sec (with seq num) write speed all = 0.002614141 MB/sec (without seq num) 2024/11/22-23:02:12.294900 140025205032512 [compaction_job.cc:1948] [default] [JOB 13448] Generated table #232745: 925015 keys, 36627094 bytes, temperature: kWarm 2024/11/22-23:02:12.294984 140025205032512 EVENT_LOG_v1 {"time_micros": 1732287732294932, "cf_name": "default", "job": 13448, "event": "table_file_creation", "file_number": 232745, "file_size": 36627094, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3513012344, "largest_seqno": 3628605580, "table_properties": {"data_size": 30658688, "index_size": 1759488, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25900420, "raw_average_key_size": 28, "raw_value_size": 108317422, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925015, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287728, "oldest_key_time": 0, "file_creation_time": 1732287728, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232745, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:02:12.331055 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232746.sst) /dev/shm/ToplingTemp/Topling-325230-cUxiSy 2024/11/22-23:02:13.469083 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232746.sst): first pass time = 1.14's, 117.943'MB/sec 2024/11/22-23:02:15.066234 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232746.sst): old prealloc_size = 147639500, real_size = 33401424 2024/11/22-23:02:15.066825 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232746.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924783 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 117.13 avg-zval = 33.95 2024/11/22-23:02:15.067569 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232746.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:02:16.001842 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232746.sst): second pass time = 1.60's, 67.856'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 40.075'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3090.913'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.287'MB/sec write SST data time = 0.13's, 289.652'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 38293.364'MB/sec zip my value time = 1.60's, unzip length = 0.108'GB zip my value throughput = 67.856'MB/sec zip pipeline throughput = 67.856'MB/sec entries = 924783 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.13 avg-zval = 33.95 usrkeys = 924783 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.13 avg-zval = 33.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924783 bytes = 3236752 } seq expand size = 6473481 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4443 value = 3.4499 dict = 1.00 all = 3.5797 } Zip/UnZip{ index = 0.0957 value = 0.2899 dict = 1.00 all = 0.2794 } ---------------------------- total value len = 0.229590 GB avg = 0.124 KB (by entry num) total key len = 0.051794 GB avg = 0.028 KB total ukey len = 0.036996 GB avg = 0.020 KB total ukey num = 0.001849798 Billion total entry num = 0.001849798 Billion write speed all = 0.005518106 MB/sec (with seq num) write speed all = 0.005227901 MB/sec (without seq num) 2024/11/22-23:02:16.045210 140025205032512 [compaction_job.cc:1948] [default] [JOB 13448] Generated table #232746: 924783 keys, 37494144 bytes, temperature: kWarm 2024/11/22-23:02:16.045271 140025205032512 EVENT_LOG_v1 {"time_micros": 1732287736045240, "cf_name": "default", "job": 13448, "event": "table_file_creation", "file_number": 232746, "file_size": 37494144, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038037, "largest_seqno": 3628603142, "table_properties": {"data_size": 31399168, "index_size": 1770880, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25893924, "raw_average_key_size": 28, "raw_value_size": 108323914, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924783, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287732, "oldest_key_time": 0, "file_creation_time": 1732287732, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232746, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:02:16.084155 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232747.sst) /dev/shm/ToplingTemp/Topling-325230-DmavCz 2024/11/22-23:02:17.332126 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232747.sst): first pass time = 1.25's, 107.552'MB/sec 2024/11/22-23:02:19.190018 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232747.sst): old prealloc_size = 147639500, real_size = 33183936 2024/11/22-23:02:19.190598 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232747.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925080 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 117.09 avg-zval = 33.72 2024/11/22-23:02:19.191351 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232747.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:02:20.322544 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232747.sst): second pass time = 1.86's, 58.321'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.13's, 32.814'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3156.110'MB/sec (index lex order gen) rebuild zvType time = 0.90's, 0.256'MB/sec write SST data time = 0.23's, 161.434'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35032.436'MB/sec zip my value time = 1.86's, unzip length = 0.108'GB zip my value throughput = 58.321'MB/sec zip pipeline throughput = 129.195'MB/sec entries = 925080 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.09 avg-zval = 33.72 usrkeys = 925080 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.09 avg-zval = 33.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925080 bytes = 3122160 } seq expand size = 6475560 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5130 value = 3.4725 dict = 1.00 all = 3.6117 } Zip/UnZip{ index = 0.0951 value = 0.2880 dict = 1.00 all = 0.2769 } ---------------------------- total value len = 0.344381 GB avg = 0.124 KB (by entry num) total key len = 0.077697 GB avg = 0.028 KB total ukey len = 0.055498 GB avg = 0.020 KB total ukey num = 0.002774878 Billion total entry num = 0.002774878 Billion write speed all = 0.008276481 MB/sec (with seq num) write speed all = 0.007841183 MB/sec (without seq num) 2024/11/22-23:02:20.364971 140025205032512 [compaction_job.cc:1948] [default] [JOB 13448] Generated table #232747: 925080 keys, 37161792 bytes, temperature: kWarm 2024/11/22-23:02:20.365035 140025205032512 EVENT_LOG_v1 {"time_micros": 1732287740365002, "cf_name": "default", "job": 13448, "event": "table_file_creation", "file_number": 232747, "file_size": 37161792, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3500816256, "largest_seqno": 3628604923, "table_properties": {"data_size": 31192624, "index_size": 1759872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25902240, "raw_average_key_size": 28, "raw_value_size": 108315520, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925080, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287736, "oldest_key_time": 0, "file_creation_time": 1732287736, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232747, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:02:20.401261 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232748.sst) /dev/shm/ToplingTemp/Topling-325230-kZaoLA 2024/11/22-23:02:21.303144 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232748.sst): first pass time = 0.90's, 148.826'MB/sec 2024/11/22-23:02:22.697348 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232748.sst): old prealloc_size = 147639500, real_size = 33430672 2024/11/22-23:02:22.697942 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232748.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924866 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 117.12 avg-zval = 33.99 2024/11/22-23:02:22.698741 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232748.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:02:23.691814 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232748.sst): second pass time = 1.39's, 77.728'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.99's, 37.733'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3052.989'MB/sec (index lex order gen) rebuild zvType time = 0.86's, 0.269'MB/sec write SST data time = 0.13's, 282.446'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 37835.153'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 77.728'MB/sec zip pipeline throughput = 77.728'MB/sec entries = 924866 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.12 avg-zval = 33.99 usrkeys = 924866 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.12 avg-zval = 33.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924866 bytes = 3237040 } seq expand size = 6474062 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5171 value = 3.4453 dict = 1.00 all = 3.5769 } Zip/UnZip{ index = 0.0951 value = 0.2903 dict = 1.00 all = 0.2796 } ---------------------------- total value len = 0.459177 GB avg = 0.124 KB (by entry num) total key len = 0.103593 GB avg = 0.028 KB total ukey len = 0.073995 GB avg = 0.020 KB total ukey num = 0.003699744 Billion total entry num = 0.003699744 Billion write speed all = 0.011034566 MB/sec (with seq num) write speed all = 0.010454221 MB/sec (without seq num) 2024/11/22-23:02:23.741934 140025205032512 [compaction_job.cc:1948] [default] [JOB 13448] Generated table #232748: 924866 keys, 37523346 bytes, temperature: kWarm 2024/11/22-23:02:23.741988 140025205032512 EVENT_LOG_v1 {"time_micros": 1732287743741959, "cf_name": "default", "job": 13448, "event": "table_file_creation", "file_number": 232748, "file_size": 37523346, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038075, "largest_seqno": 3628606802, "table_properties": {"data_size": 31440512, "index_size": 1758784, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25896248, "raw_average_key_size": 28, "raw_value_size": 108321564, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924866, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287740, "oldest_key_time": 0, "file_creation_time": 1732287740, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232748, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:02:23.780169 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232749.sst) /dev/shm/ToplingTemp/Topling-325230-5oSKcx 2024/11/22-23:02:25.194563 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232749.sst): first pass time = 1.41's, 94.897'MB/sec 2024/11/22-23:02:26.818753 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232749.sst): old prealloc_size = 147639500, real_size = 32325920 2024/11/22-23:02:26.819324 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232749.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930338 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0303 GB avg-val = 116.27 avg-zval = 32.59 2024/11/22-23:02:26.820119 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232749.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:02:27.771638 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232749.sst): second pass time = 1.62's, 66.637'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 38.236'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3275.999'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.283'MB/sec write SST data time = 0.13's, 280.643'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 24515.151'MB/sec zip my value time = 1.62's, unzip length = 0.108'GB zip my value throughput = 66.637'MB/sec zip pipeline throughput = 66.637'MB/sec entries = 930338 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.27 avg-zval = 32.59 usrkeys = 930338 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.27 avg-zval = 32.59 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930338 bytes = 3256192 } seq expand size = 6512366 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5112 value = 3.5672 dict = 1.00 all = 3.6839 } Zip/UnZip{ index = 0.0951 value = 0.2803 dict = 1.00 all = 0.2715 } ---------------------------- total value len = 0.573857 GB avg = 0.124 KB (by entry num) total key len = 0.129642 GB avg = 0.028 KB total ukey len = 0.092602 GB avg = 0.020 KB total ukey num = 0.004630082 Billion total entry num = 0.004630082 Billion write speed all = 0.013792846 MB/sec (with seq num) write speed all = 0.013066626 MB/sec (without seq num) 2024/11/22-23:02:27.811360 140025205032512 [compaction_job.cc:1948] [default] [JOB 13448] Generated table #232749: 930338 keys, 36433790 bytes, temperature: kWarm 2024/11/22-23:02:27.811437 140025205032512 EVENT_LOG_v1 {"time_micros": 1732287747811390, "cf_name": "default", "job": 13448, "event": "table_file_creation", "file_number": 232749, "file_size": 36433790, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038106, "largest_seqno": 3628606566, "table_properties": {"data_size": 30322960, "index_size": 1770176, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26049464, "raw_average_key_size": 28, "raw_value_size": 108168364, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930338, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287743, "oldest_key_time": 0, "file_creation_time": 1732287743, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232749, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:02:27.849666 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232750.sst) /dev/shm/ToplingTemp/Topling-325230-2kdwuz 2024/11/22-23:02:29.470190 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232750.sst): first pass time = 1.62's, 82.826'MB/sec 2024/11/22-23:02:31.062372 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232750.sst): old prealloc_size = 147639500, real_size = 31151088 2024/11/22-23:02:31.062960 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232750.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924797 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0291 GB avg-val = 117.13 avg-zval = 31.52 2024/11/22-23:02:31.063722 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232750.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:02:32.114572 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232750.sst): second pass time = 1.59's, 68.064'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.05's, 33.496'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3191.937'MB/sec (index lex order gen) rebuild zvType time = 0.92's, 0.250'MB/sec write SST data time = 0.13's, 275.765'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 26708.068'MB/sec zip my value time = 1.59's, unzip length = 0.108'GB zip my value throughput = 68.064'MB/sec zip pipeline throughput = 68.064'MB/sec entries = 924797 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.13 avg-zval = 31.52 usrkeys = 924797 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.13 avg-zval = 31.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924797 bytes = 3236800 } seq expand size = 6473579 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0352 GB } UnZip/Zip{ index = 10.4445 value = 3.7162 dict = 1.00 all = 3.8083 } Zip/UnZip{ index = 0.0957 value = 0.2691 dict = 1.00 all = 0.2626 } ---------------------------- total value len = 0.688654 GB avg = 0.124 KB (by entry num) total key len = 0.155537 GB avg = 0.028 KB total ukey len = 0.111098 GB avg = 0.020 KB total ukey num = 0.005554879 Billion total entry num = 0.005554879 Billion write speed all = 0.016549838 MB/sec (with seq num) write speed all = 0.015678638 MB/sec (without seq num) 2024/11/22-23:02:32.151437 140025205032512 [compaction_job.cc:1948] [default] [JOB 13448] Generated table #232750: 924797 keys, 35243772 bytes, temperature: kWarm 2024/11/22-23:02:32.151499 140025205032512 EVENT_LOG_v1 {"time_micros": 1732287752151468, "cf_name": "default", "job": 13448, "event": "table_file_creation", "file_number": 232750, "file_size": 35243772, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3435169332, "largest_seqno": 3628606623, "table_properties": {"data_size": 29148832, "index_size": 1770880, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25894316, "raw_average_key_size": 28, "raw_value_size": 108323438, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924797, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287747, "oldest_key_time": 0, "file_creation_time": 1732287747, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232750, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:02:32.192899 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232751.sst) /dev/shm/ToplingTemp/Topling-325230-SBRJjA 2024/11/22-23:02:33.546029 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232751.sst): first pass time = 1.35's, 99.194'MB/sec 2024/11/22-23:02:35.166558 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232751.sst): old prealloc_size = 147639500, real_size = 32891008 2024/11/22-23:02:35.167254 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232751.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925144 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 117.08 avg-zval = 33.40 2024/11/22-23:02:35.168064 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232751.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:02:36.381925 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232751.sst): second pass time = 1.62's, 66.869'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.22's, 30.431'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2610.224'MB/sec (index lex order gen) rebuild zvType time = 0.95's, 0.243'MB/sec write SST data time = 0.26's, 141.425'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35702.563'MB/sec zip my value time = 1.62's, unzip length = 0.108'GB zip my value throughput = 66.869'MB/sec zip pipeline throughput = 66.869'MB/sec entries = 925144 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.08 avg-zval = 33.40 usrkeys = 925144 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.08 avg-zval = 33.40 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925144 bytes = 3238016 } seq expand size = 6476008 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5141 value = 3.5053 dict = 1.00 all = 3.6290 } Zip/UnZip{ index = 0.0951 value = 0.2853 dict = 1.00 all = 0.2756 } ---------------------------- total value len = 0.803444 GB avg = 0.124 KB (by entry num) total key len = 0.181441 GB avg = 0.028 KB total ukey len = 0.129600 GB avg = 0.020 KB total ukey num = 0.006480023 Billion total entry num = 0.006480023 Billion write speed all = 0.019306437 MB/sec (with seq num) write speed all = 0.018290228 MB/sec (without seq num) 2024/11/22-23:02:36.424421 140025205032512 [compaction_job.cc:1948] [default] [JOB 13448] Generated table #232751: 925144 keys, 36984420 bytes, temperature: kWarm 2024/11/22-23:02:36.424499 140025205032512 EVENT_LOG_v1 {"time_micros": 1732287756424456, "cf_name": "default", "job": 13448, "event": "table_file_creation", "file_number": 232751, "file_size": 36984420, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038136, "largest_seqno": 3628605653, "table_properties": {"data_size": 30899760, "index_size": 1759808, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25904032, "raw_average_key_size": 28, "raw_value_size": 108313780, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925144, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287752, "oldest_key_time": 0, "file_creation_time": 1732287752, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232751, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:02:36.500714 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232752.sst) /dev/shm/ToplingTemp/Topling-325230-rUcleB 2024/11/22-23:02:37.927533 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232752.sst): first pass time = 1.43's, 94.071'MB/sec 2024/11/22-23:02:39.535031 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232752.sst): old prealloc_size = 147639500, real_size = 32696224 2024/11/22-23:02:39.535610 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232752.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924820 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0307 GB avg-val = 117.13 avg-zval = 33.19 2024/11/22-23:02:39.536359 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232752.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:02:40.433198 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232752.sst): second pass time = 1.61's, 67.416'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 40.832'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3172.210'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.299'MB/sec write SST data time = 0.13's, 292.100'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 26366.164'MB/sec zip my value time = 1.61's, unzip length = 0.108'GB zip my value throughput = 67.416'MB/sec zip pipeline throughput = 106.632'MB/sec entries = 924820 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.13 avg-zval = 33.19 usrkeys = 924820 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.13 avg-zval = 33.19 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924820 bytes = 3121280 } seq expand size = 6473740 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.4448 value = 3.5291 dict = 1.00 all = 3.6598 } Zip/UnZip{ index = 0.0957 value = 0.2834 dict = 1.00 all = 0.2732 } ---------------------------- total value len = 0.918241 GB avg = 0.124 KB (by entry num) total key len = 0.207336 GB avg = 0.028 KB total ukey len = 0.148097 GB avg = 0.020 KB total ukey num = 0.007404843 Billion total entry num = 0.007404843 Billion write speed all = 0.022062626 MB/sec (with seq num) write speed all = 0.020901476 MB/sec (without seq num) 2024/11/22-23:02:40.470480 140025205032512 [compaction_job.cc:1948] [default] [JOB 13448] Generated table #232752: 924820 keys, 36673394 bytes, temperature: kWarm 2024/11/22-23:02:40.470544 140025205032512 EVENT_LOG_v1 {"time_micros": 1732287760470509, "cf_name": "default", "job": 13448, "event": "table_file_creation", "file_number": 232752, "file_size": 36673394, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3499102480, "largest_seqno": 3628606025, "table_properties": {"data_size": 30693968, "index_size": 1770880, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25894960, "raw_average_key_size": 28, "raw_value_size": 108322908, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924820, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287756, "oldest_key_time": 0, "file_creation_time": 1732287756, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232752, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:02:40.518460 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232753.sst) /dev/shm/ToplingTemp/Topling-325230-k7R2iA 2024/11/22-23:02:41.347719 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232753.sst): first pass time = 0.83's, 161.860'MB/sec 2024/11/22-23:02:42.724203 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232753.sst): old prealloc_size = 147639500, real_size = 33904992 2024/11/22-23:02:42.724775 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232753.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925919 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0319 GB avg-val = 116.96 avg-zval = 34.46 2024/11/22-23:02:42.725600 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232753.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:02:43.653802 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232753.sst): second pass time = 1.38's, 78.709'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 41.008'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3200.477'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.290'MB/sec write SST data time = 0.13's, 289.967'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 36072.041'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.709'MB/sec zip pipeline throughput = 78.709'MB/sec entries = 925919 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.96 avg-zval = 34.46 usrkeys = 925919 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.96 avg-zval = 34.46 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925919 bytes = 3356464 } seq expand size = 6481433 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.5119 value = 3.3935 dict = 1.00 all = 3.5209 } Zip/UnZip{ index = 0.0951 value = 0.2947 dict = 1.00 all = 0.2840 } ---------------------------- total value len = 1.033014 GB avg = 0.124 KB (by entry num) total key len = 0.233261 GB avg = 0.028 KB total ukey len = 0.166615 GB avg = 0.020 KB total ukey num = 0.008330762 Billion total entry num = 0.008330762 Billion write speed all = 0.024818929 MB/sec (with seq num) write speed all = 0.023512670 MB/sec (without seq num) 2024/11/22-23:02:43.694801 140025205032512 [compaction_job.cc:1948] [default] [JOB 13448] Generated table #232753: 925919 keys, 38120042 bytes, temperature: kWarm 2024/11/22-23:02:43.694866 140025205032512 EVENT_LOG_v1 {"time_micros": 1732287763694833, "cf_name": "default", "job": 13448, "event": "table_file_creation", "file_number": 232753, "file_size": 38120042, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868609, "largest_seqno": 3628604753, "table_properties": {"data_size": 31911696, "index_size": 1761664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25925732, "raw_average_key_size": 28, "raw_value_size": 108292038, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925919, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287760, "oldest_key_time": 0, "file_creation_time": 1732287760, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232753, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:02:43.734090 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232754.sst) /dev/shm/ToplingTemp/Topling-325230-ko1xXA 2024/11/22-23:02:44.540259 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232754.sst): first pass time = 0.81's, 166.497'MB/sec 2024/11/22-23:02:46.060461 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232754.sst): old prealloc_size = 147639500, real_size = 36910880 2024/11/22-23:02:46.061019 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232754.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926492 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0349 GB avg-val = 116.87 avg-zval = 37.69 2024/11/22-23:02:46.061264 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232754.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:02:46.727973 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232754.sst): second pass time = 1.52's, 71.254'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.67's, 57.501'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3290.093'MB/sec (index lex order gen) rebuild zvType time = 0.54's, 0.428'MB/sec write SST data time = 0.13's, 305.537'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 26380.071'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 71.254'MB/sec zip pipeline throughput = 71.254'MB/sec entries = 926492 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.87 avg-zval = 37.69 usrkeys = 926492 avg-key = 20.00 avg-zkey = 1.90 avg-val = 118.00 avg-zval = 37.69 TagRS{ kinds = RS_Key0_TagN bytes = 144880 } TagArray{ size = 149750 bytes = 467984 } seq expand size = 1048250 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0349 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.5142 value = 3.1010 dict = 1.00 all = 3.4969 } Zip/UnZip{ index = 0.0951 value = 0.3225 dict = 1.00 all = 0.2860 } ---------------------------- total value len = 1.142339 GB avg = 0.123 KB (by entry num) total key len = 0.259203 GB avg = 0.028 KB total ukey len = 0.185145 GB avg = 0.020 KB total ukey num = 0.009257254 Billion total entry num = 0.009257254 Billion write speed all = 0.027468481 MB/sec (with seq num) write speed all = 0.026017035 MB/sec (without seq num) 2024/11/22-23:02:46.767479 140025205032512 [compaction_job.cc:1948] [default] [JOB 13448] Generated table #232754: 926492 keys, 38381843 bytes, temperature: kWarm 2024/11/22-23:02:46.767540 140025205032512 EVENT_LOG_v1 {"time_micros": 1732287766767509, "cf_name": "default", "job": 13448, "event": "table_file_creation", "file_number": 232754, "file_size": 38381843, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700833, "table_properties": {"data_size": 34916688, "index_size": 1762368, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25941776, "raw_average_key_size": 28, "raw_value_size": 108276016, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926492, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287763, "oldest_key_time": 0, "file_creation_time": 1732287763, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232754, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:02:46.805486 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232755.sst) /dev/shm/ToplingTemp/Topling-325230-4IyPNA 2024/11/22-23:02:47.573736 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232755.sst): first pass time = 0.77's, 174.715'MB/sec 2024/11/22-23:02:49.120842 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232755.sst): old prealloc_size = 147639500, real_size = 36484608 2024/11/22-23:02:49.121395 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232755.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927067 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.78 avg-zval = 37.20 2024/11/22-23:02:49.121438 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232755.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:02:49.761659 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232755.sst): second pass time = 1.55's, 70.030'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 58.275'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3297.119'MB/sec (index lex order gen) rebuild zvType time = 0.51's, 0.450'MB/sec write SST data time = 0.13's, 297.522'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33201.475'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 70.030'MB/sec zip pipeline throughput = 70.030'MB/sec entries = 927067 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.78 avg-zval = 37.20 usrkeys = 927067 avg-key = 20.00 avg-zkey = 1.90 avg-val = 116.78 avg-zval = 37.20 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5119 value = 3.1390 dict = 1.00 all = 3.5942 } Zip/UnZip{ index = 0.0951 value = 0.3186 dict = 1.00 all = 0.2782 } ---------------------------- total value len = 1.250598 GB avg = 0.123 KB (by entry num) total key len = 0.285161 GB avg = 0.028 KB total ukey len = 0.203686 GB avg = 0.020 KB total ukey num = 0.010184321 Billion total entry num = 0.010184321 Billion write speed all = 0.030097194 MB/sec (with seq num) write speed all = 0.028500488 MB/sec (without seq num) 2024/11/22-23:02:49.798897 140025205032512 [compaction_job.cc:1948] [default] [JOB 13448] Generated table #232755: 927067 keys, 37342612 bytes, temperature: kWarm 2024/11/22-23:02:49.798961 140025205032512 EVENT_LOG_v1 {"time_micros": 1732287769798928, "cf_name": "default", "job": 13448, "event": "table_file_creation", "file_number": 232755, "file_size": 37342612, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34488816, "index_size": 1763840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25957876, "raw_average_key_size": 28, "raw_value_size": 108259898, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927067, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287766, "oldest_key_time": 0, "file_creation_time": 1732287766, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232755, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:02:49.834467 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232756.sst) /dev/shm/ToplingTemp/Topling-325230-IDQs4A 2024/11/22-23:02:50.596826 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232756.sst): first pass time = 0.76's, 176.064'MB/sec 2024/11/22-23:02:51.996682 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232756.sst): old prealloc_size = 147639500, real_size = 35704672 2024/11/22-23:02:51.997242 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232756.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923155 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0337 GB avg-val = 117.39 avg-zval = 36.52 2024/11/22-23:02:51.997661 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232756.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:02:52.786172 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232756.sst): second pass time = 1.40's, 77.476'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.79's, 48.420'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3271.379'MB/sec (index lex order gen) rebuild zvType time = 0.66's, 0.349'MB/sec write SST data time = 0.13's, 297.293'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34368.767'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.476'MB/sec zip pipeline throughput = 77.476'MB/sec entries = 923155 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.39 avg-zval = 36.52 usrkeys = 923155 avg-key = 20.00 avg-zkey = 1.90 avg-val = 120.80 avg-zval = 36.52 TagRS{ kinds = RS_Key0_TagN bytes = 144360 } TagArray{ size = 449193 bytes = 1516048 } seq expand size = 3144351 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0337 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.5110 value = 3.2141 dict = 1.00 all = 3.5111 } Zip/UnZip{ index = 0.0951 value = 0.3111 dict = 1.00 all = 0.2848 } ---------------------------- total value len = 1.362112 GB avg = 0.123 KB (by entry num) total key len = 0.311009 GB avg = 0.028 KB total ukey len = 0.222150 GB avg = 0.020 KB total ukey num = 0.011107476 Billion total entry num = 0.011107476 Billion write speed all = 0.032787219 MB/sec (with seq num) write speed all = 0.031045884 MB/sec (without seq num) 2024/11/22-23:02:52.827181 140025205032512 [compaction_job.cc:1948] [default] [JOB 13448] Generated table #232756: 923155 keys, 38226803 bytes, temperature: kWarm 2024/11/22-23:02:52.827245 140025205032512 EVENT_LOG_v1 {"time_micros": 1732287772827211, "cf_name": "default", "job": 13448, "event": "table_file_creation", "file_number": 232756, "file_size": 38226803, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3397633721, "table_properties": {"data_size": 33717136, "index_size": 1756544, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25848340, "raw_average_key_size": 28, "raw_value_size": 108369502, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923155, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287769, "oldest_key_time": 0, "file_creation_time": 1732287769, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232756, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:02:52.863967 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232757.sst) /dev/shm/ToplingTemp/Topling-325230-kDWvOy 2024/11/22-23:02:53.768263 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232757.sst): first pass time = 0.90's, 148.430'MB/sec 2024/11/22-23:02:55.110864 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232757.sst): old prealloc_size = 147639500, real_size = 33405264 2024/11/22-23:02:55.111407 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232757.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925335 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 117.05 avg-zval = 33.94 2024/11/22-23:02:55.112170 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232757.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:02:56.048675 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232757.sst): second pass time = 1.34's, 80.733'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.986'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3375.120'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.286'MB/sec write SST data time = 0.13's, 289.419'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33642.188'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.733'MB/sec zip pipeline throughput = 80.733'MB/sec entries = 925335 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.05 avg-zval = 33.94 usrkeys = 925335 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.05 avg-zval = 33.94 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925335 bytes = 3238688 } seq expand size = 6477345 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4449 value = 3.4491 dict = 1.00 all = 3.5792 } Zip/UnZip{ index = 0.0957 value = 0.2899 dict = 1.00 all = 0.2794 } ---------------------------- total value len = 1.476898 GB avg = 0.123 KB (by entry num) total key len = 0.336919 GB avg = 0.028 KB total ukey len = 0.240656 GB avg = 0.020 KB total ukey num = 0.012032811 Billion total entry num = 0.012032811 Billion write speed all = 0.035542070 MB/sec (with seq num) write speed all = 0.033655789 MB/sec (without seq num) 2024/11/22-23:02:56.087960 140025205032512 [compaction_job.cc:1948] [default] [JOB 13448] Generated table #232757: 925335 keys, 37499136 bytes, temperature: kWarm 2024/11/22-23:02:56.088021 140025205032512 EVENT_LOG_v1 {"time_micros": 1732287776087988, "cf_name": "default", "job": 13448, "event": "table_file_creation", "file_number": 232757, "file_size": 37499136, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3397633722, "largest_seqno": 3628606648, "table_properties": {"data_size": 31401920, "index_size": 1771840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25909380, "raw_average_key_size": 28, "raw_value_size": 108308426, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925335, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287772, "oldest_key_time": 0, "file_creation_time": 1732287772, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232757, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:02:56.124425 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232758.sst) /dev/shm/ToplingTemp/Topling-325230-1hO8Zw 2024/11/22-23:02:57.034209 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232758.sst): first pass time = 0.91's, 147.533'MB/sec 2024/11/22-23:02:58.402412 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232758.sst): old prealloc_size = 147639500, real_size = 33863680 2024/11/22-23:02:58.402991 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232758.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931138 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0318 GB avg-val = 116.14 avg-zval = 34.20 2024/11/22-23:02:58.404713 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232758.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:02:59.326460 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232758.sst): second pass time = 1.37's, 79.079'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 41.222'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3192.128'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.293'MB/sec write SST data time = 0.13's, 294.769'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 38521.469'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.079'MB/sec zip pipeline throughput = 79.079'MB/sec entries = 931138 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.14 avg-zval = 34.20 usrkeys = 931138 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.14 avg-zval = 34.20 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931138 bytes = 3375392 } seq expand size = 6517966 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.4448 value = 3.3957 dict = 1.00 all = 3.5237 } Zip/UnZip{ index = 0.0957 value = 0.2945 dict = 1.00 all = 0.2838 } ---------------------------- total value len = 1.591562 GB avg = 0.123 KB (by entry num) total key len = 0.362991 GB avg = 0.028 KB total ukey len = 0.259279 GB avg = 0.020 KB total ukey num = 0.012963949 Billion total entry num = 0.012963949 Billion write speed all = 0.038297353 MB/sec (with seq num) write speed all = 0.036265236 MB/sec (without seq num) 2024/11/22-23:02:59.364547 140025205032512 [compaction_job.cc:1948] [default] [JOB 13448] Generated table #232758: 931138 keys, 38090482 bytes, temperature: kWarm 2024/11/22-23:02:59.364608 140025205032512 EVENT_LOG_v1 {"time_micros": 1732287779364577, "cf_name": "default", "job": 13448, "event": "table_file_creation", "file_number": 232758, "file_size": 38090482, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701386, "largest_seqno": 3628606674, "table_properties": {"data_size": 31847728, "index_size": 1782976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26071864, "raw_average_key_size": 28, "raw_value_size": 108145932, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931138, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287776, "oldest_key_time": 0, "file_creation_time": 1732287776, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232758, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:02:59.403143 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232759.sst) /dev/shm/ToplingTemp/Topling-325230-g4cpcy 2024/11/22-23:03:00.291518 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232759.sst): first pass time = 0.89's, 151.089'MB/sec 2024/11/22-23:03:01.643016 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232759.sst): old prealloc_size = 147639500, real_size = 35233408 2024/11/22-23:03:01.643549 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232759.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929974 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0332 GB avg-val = 116.32 avg-zval = 35.72 2024/11/22-23:03:01.644328 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232759.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:03:02.520650 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232759.sst): second pass time = 1.35's, 80.082'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 44.826'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3471.070'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.310'MB/sec write SST data time = 0.13's, 308.146'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 23792.232'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.082'MB/sec zip pipeline throughput = 80.082'MB/sec entries = 929974 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.32 avg-zval = 35.72 usrkeys = 929974 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.32 avg-zval = 35.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929974 bytes = 3254928 } seq expand size = 6509818 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.4467 value = 3.2564 dict = 1.00 all = 3.4117 } Zip/UnZip{ index = 0.0957 value = 0.3071 dict = 1.00 all = 0.2931 } ---------------------------- total value len = 1.706250 GB avg = 0.123 KB (by entry num) total key len = 0.389030 GB avg = 0.028 KB total ukey len = 0.277878 GB avg = 0.020 KB total ukey num = 0.013893923 Billion total entry num = 0.013893923 Billion write speed all = 0.041052191 MB/sec (with seq num) write speed all = 0.038874435 MB/sec (without seq num) 2024/11/22-23:03:02.560175 140025205032512 [compaction_job.cc:1948] [default] [JOB 13448] Generated table #232759: 929974 keys, 39340428 bytes, temperature: kWarm 2024/11/22-23:03:02.560242 140025205032512 EVENT_LOG_v1 {"time_micros": 1732287782560208, "cf_name": "default", "job": 13448, "event": "table_file_creation", "file_number": 232759, "file_size": 39340428, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3628606699, "table_properties": {"data_size": 33220336, "index_size": 1780416, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26039272, "raw_average_key_size": 28, "raw_value_size": 108178556, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929974, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287779, "oldest_key_time": 0, "file_creation_time": 1732287779, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232759, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:03:02.605871 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232760.sst) /dev/shm/ToplingTemp/Topling-325230-DF2PNy 2024/11/22-23:03:02.686050 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232760.sst): first pass time = 0.08's, 237.414'MB/sec 2024/11/22-23:03:02.861919 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232760.sst): old prealloc_size = 147639500, real_size = 5620400 2024/11/22-23:03:02.862060 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232760.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 130572 prefix = 4 raw-key = 0.0021 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.0154 GB zip-val = 0.0053 GB avg-val = 117.71 avg-zval = 40.86 2024/11/22-23:03:02.862191 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232760.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.019 GB 2024/11/22-23:03:02.985306 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00/232760.sst): second pass time = 0.18's, 87.670'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.12's, 50.043'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2229.796'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.313'MB/sec write SST data time = 0.02's, 324.943'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.161'MB, throughput = 5723.309'MB/sec zip my value time = 0.18's, unzip length = 0.015'GB zip my value throughput = 87.670'MB/sec zip pipeline throughput = 87.670'MB/sec entries = 130572 avg-key = 28.00 avg-zkey = 1.94 avg-val = 117.71 avg-zval = 40.86 usrkeys = 130572 avg-key = 20.00 avg-zkey = 1.94 avg-val = 124.71 avg-zval = 40.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 130572 bytes = 424368 } seq expand size = 914004 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0026 GB value = 0.0154 GB dict = 0.16 MB all = 0.0190 GB } __ZipSize{ index = 0.0003 GB value = 0.0053 GB dict = 0.16 MB all = 0.0062 GB } UnZip/Zip{ index = 10.3248 value = 2.8811 dict = 1.00 all = 3.0815 } Zip/UnZip{ index = 0.0969 value = 0.3471 dict = 1.00 all = 0.3245 } ---------------------------- total value len = 1.722534 GB avg = 0.123 KB (by entry num) total key len = 0.392686 GB avg = 0.028 KB total ukey len = 0.280490 GB avg = 0.020 KB total ukey num = 0.014024495 Billion total entry num = 0.014024495 Billion write speed all = 0.041442480 MB/sec (with seq num) write speed all = 0.039244279 MB/sec (without seq num) 2024/11/22-23:03:02.997220 140025205032512 [compaction_job.cc:1948] [default] [JOB 13448] Generated table #232760: 130572 keys, 6174075 bytes, temperature: kWarm 2024/11/22-23:03:02.997274 140025205032512 EVENT_LOG_v1 {"time_micros": 1732287782997245, "cf_name": "default", "job": 13448, "event": "table_file_creation", "file_number": 232760, "file_size": 6174075, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3587701383, "largest_seqno": 3627102341, "table_properties": {"data_size": 5334624, "index_size": 252928, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3656016, "raw_average_key_size": 28, "raw_value_size": 15369408, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 130572, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287782, "oldest_key_time": 0, "file_creation_time": 1732287782, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232760, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:03:03.009834 140025205032512 [compaction_job.cc:703] job-13448: subcompact[0], size: 0.567317 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13448/att-00 [232745,232746,232747,232748,232749,232750,232751,232752,232753,232754,232755,232756,232757,232758,232759,232760] 2024/11/22-23:03:03.110322 140025205032512 (Original Log Time 2024/11/22-23:02:07.642980) EVENT_LOG_v1 {"time_micros": 1732287727642940, "job": 13448, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [232734], "files_L3": [232672, 232673, 232674, 232675, 232676, 232677, 232678, 232679, 232680, 232681, 232682, 232683, 232684, 232685, 232697], "score": 0.999776, "input_data_size": 591449514, "oldest_snapshot_seqno": 3628634313} 2024/11/22-23:03:03.110325 140025205032512 (Original Log Time 2024/11/22-23:03:03.109504) [compaction_job.cc:2022] [default] [JOB 13448] Compacted 1@2 + 15@3 files to L3 => 567317173 bytes 2024/11/22-23:03:03.110326 140025205032512 (Original Log Time 2024/11/22-23:03:03.110254) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 16 0 0 0] max score 0.02, MB/sec: 10.7 rd, 10.2 wr, level 3, files in(1, 15) out(16 +0 blob) MB in(27.1, 537.0 +0.0 blob) out(541.0 +0.0 blob), read-write-amplify(40.8) write-amplify(20.0) OK, records in: 14126399, records dropped: 101904 output_compression: Snappy 2024/11/22-23:03:03.110331 140025205032512 (Original Log Time 2024/11/22-23:03:03.110303) EVENT_LOG_v1 {"time_micros": 1732287783110278, "cf": "default", "job": 13448, "event": "compaction_finished", "compaction_time_micros": 55366861, "compaction_time_cpu_micros": 24428224, "output_level": 3, "num_output_files": 16, "total_output_size": 567317173, "num_input_records": 14126399, "num_output_records": 14024495, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 372392964, "file_range_sync_nanos": 0, "file_fsync_nanos": 14544, "file_prepare_write_nanos": 57688, "lsm_state": [0, 0, 0, 16, 0, 0, 0]} 2024/11/22-23:03:03.242197 140025205032512 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732287727,"executesMs":55595,"compactionJobId":13448,"attempt":0,"compactionInputRawBytes":2046762914,"compactionInputZipBytes":591449514,"compactionOutputRawBytes":0,"compactionOutputZipBytes":567317173,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-998d70e145173dbed80a2e5f7fcd2b12-494932f64ee9fa31-00"}}, response = 2024/11/22-23:03:03.242272 140025205032512 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-13448/att-00: olev 3, work 55.595 s, result 4.873 ms, install 0.845 ms, input{raw 1.906 GiB zip 564.050 MiB}