2024/11/23-00:47:43.090528 140024953251392 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00: fork to child time = 0.008867 sec 2024/11/23-00:47:43.091356 140024953251392 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00 2024/11/23-00:47:43.098164 140024953251392 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00 2024/11/23-00:47:43.099292 140024953251392 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-00:47:43.099535 140024953251392 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-00:47:43.144794 140024953251392 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-15606/att-00/MANIFEST-206347 2024/11/23-00:47:43.164578 140024953251392 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-15606/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 237098, last_sequence is 3685544987, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 236844 2024/11/23-00:47:43.164584 140024953251392 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-00:47:43.164724 140024953251392 [version_set.cc:5721] Creating manifest 237098 2024/11/23-00:47:43.198706 140024953251392 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-00:47:43.219881 140024953251392 [compaction_job.cc:2437] [default] [JOB 15606] Compacting 1@2 + 17@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-00:47:43.219900 140024953251392 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 11689 Base level 2, inputs: [237095(27MB)], [237063(37MB) 237064(36MB) 237065(35MB) 237066(36MB) 237067(35MB) 237068(34MB) 237069(34MB) 237070(7419KB) 237026(35MB) 237027(34MB) 237028(34MB) 237029(34MB) 237030(34MB) 237031(34MB) 237032(35MB) 237033(35MB) 237046(37MB)] 2024/11/23-00:47:43.225795 140024953251392 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-00:47:43.225826 140024953251392 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-00:47:43.225846 140024953251392 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-00:47:43.225852 140024953251392 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-00:47:43.225863 140024953251392 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-00:47:43.699697 140024953251392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237099.sst) /dev/shm/ToplingTemp/Topling-355990-1x9zyw 2024/11/23-00:47:44.259129 140024953251392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237099.sst): first pass time = 0.56's, 239.935'MB/sec 2024/11/23-00:47:46.698077 140024953251392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237099.sst): old prealloc_size = 147639500, real_size = 34575184 2024/11/23-00:47:46.698787 140024953251392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237099.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928194 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1082 GB zip-val = 0.0326 GB avg-val = 116.60 avg-zval = 35.11 2024/11/23-00:47:46.699675 140024953251392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237099.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:47:48.224843 140024953251392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237099.sst): second pass time = 2.41's, 44.918'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.53's, 25.412'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2547.254'MB/sec (index lex order gen) rebuild zvType time = 1.26's, 0.184'MB/sec write SST data time = 0.27's, 145.647'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34951.826'MB/sec zip my value time = 2.41's, unzip length = 0.108'GB zip my value throughput = 44.918'MB/sec zip pipeline throughput = 80.284'MB/sec entries = 928194 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.60 avg-zval = 35.11 usrkeys = 928194 avg-key = 20.00 avg-zkey = 1.89 avg-val = 123.60 avg-zval = 35.11 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928194 bytes = 3364720 } seq expand size = 6497358 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0326 GB dict = 1.08 MB all = 0.0388 GB } UnZip/Zip{ index = 10.5815 value = 3.3211 dict = 1.00 all = 3.4595 } Zip/UnZip{ index = 0.0945 value = 0.3011 dict = 1.00 all = 0.2891 } ---------------------------- total value len = 0.114726 GB avg = 0.124 KB (by entry num) total key len = 0.025989 GB avg = 0.028 KB total ukey len = 0.018564 GB avg = 0.020 KB total ukey num = 0.000928194 Billion total entry num = 0.000928194 Billion write speed all = 0.002454687 MB/sec (with seq num) write speed all = 0.002325153 MB/sec (without seq num) 2024/11/23-00:47:48.261701 140024953251392 [compaction_job.cc:1948] [default] [JOB 15606] Generated table #237099: 928194 keys, 38796984 bytes, temperature: kWarm 2024/11/23-00:47:48.261821 140024953251392 EVENT_LOG_v1 {"time_micros": 1732294068261743, "cf_name": "default", "job": 15606, "event": "table_file_creation", "file_number": 237099, "file_size": 38796984, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868609, "largest_seqno": 3685525409, "table_properties": {"data_size": 32588608, "index_size": 1754368, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25989432, "raw_average_key_size": 28, "raw_value_size": 108228436, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928194, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732294063, "oldest_key_time": 0, "file_creation_time": 1732294063, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 237099, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:47:48.336371 140024953251392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237100.sst) /dev/shm/ToplingTemp/Topling-355990-PL8iBs 2024/11/23-00:47:49.371169 140024953251392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237100.sst): first pass time = 1.03's, 129.710'MB/sec 2024/11/23-00:47:51.342612 140024953251392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237100.sst): old prealloc_size = 147639500, real_size = 36945984 2024/11/23-00:47:51.343298 140024953251392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237100.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926028 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0350 GB avg-val = 116.94 avg-zval = 37.74 2024/11/23-00:47:51.343584 140024953251392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237100.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:47:52.798652 140024953251392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237100.sst): second pass time = 1.97's, 54.951'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.46's, 26.486'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2648.962'MB/sec (index lex order gen) rebuild zvType time = 1.08's, 0.215'MB/sec write SST data time = 0.38's, 102.530'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34625.997'MB/sec zip my value time = 1.97's, unzip length = 0.108'GB zip my value throughput = 54.951'MB/sec zip pipeline throughput = 54.951'MB/sec entries = 926028 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.94 avg-zval = 37.74 usrkeys = 926028 avg-key = 20.00 avg-zkey = 1.90 avg-val = 118.37 avg-zval = 37.74 TagRS{ kinds = RS_Key0_TagN bytes = 144800 } TagArray{ size = 189121 bytes = 614656 } seq expand size = 1323847 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0350 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.5139 value = 3.0982 dict = 1.00 all = 3.4804 } Zip/UnZip{ index = 0.0951 value = 0.3228 dict = 1.00 all = 0.2873 } ---------------------------- total value len = 0.224339 GB avg = 0.121 KB (by entry num) total key len = 0.051918 GB avg = 0.028 KB total ukey len = 0.037084 GB avg = 0.020 KB total ukey num = 0.001854222 Billion total entry num = 0.001854222 Billion write speed all = 0.004818740 MB/sec (with seq num) write speed all = 0.004559995 MB/sec (without seq num) 2024/11/23-00:47:52.835790 140024953251392 [compaction_job.cc:1948] [default] [JOB 15606] Generated table #237100: 926028 keys, 38564183 bytes, temperature: kWarm 2024/11/23-00:47:52.835873 140024953251392 EVENT_LOG_v1 {"time_micros": 1732294072835827, "cf_name": "default", "job": 15606, "event": "table_file_creation", "file_number": 237100, "file_size": 38564183, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700833, "table_properties": {"data_size": 34952752, "index_size": 1761536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25928784, "raw_average_key_size": 28, "raw_value_size": 108289068, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926028, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732294068, "oldest_key_time": 0, "file_creation_time": 1732294068, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 237100, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:47:52.889570 140024953251392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237101.sst) /dev/shm/ToplingTemp/Topling-355990-yDd1Rt 2024/11/23-00:47:53.726322 140024953251392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237101.sst): first pass time = 0.84's, 160.411'MB/sec 2024/11/23-00:47:55.488958 140024953251392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237101.sst): old prealloc_size = 147639500, real_size = 36468560 2024/11/23-00:47:55.489496 140024953251392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237101.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927062 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.78 avg-zval = 37.20 2024/11/23-00:47:55.489540 140024953251392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237101.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:47:56.133939 140024953251392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237101.sst): second pass time = 1.76's, 61.454'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 57.874'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3396.174'MB/sec (index lex order gen) rebuild zvType time = 0.52's, 0.448'MB/sec write SST data time = 0.13's, 294.120'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36411.299'MB/sec zip my value time = 1.76's, unzip length = 0.108'GB zip my value throughput = 61.454'MB/sec zip pipeline throughput = 66.074'MB/sec entries = 927062 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.78 avg-zval = 37.20 usrkeys = 927062 avg-key = 20.00 avg-zkey = 1.89 avg-val = 116.78 avg-zval = 37.20 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5814 value = 3.1394 dict = 1.00 all = 3.5958 } Zip/UnZip{ index = 0.0945 value = 0.3185 dict = 1.00 all = 0.2781 } ---------------------------- total value len = 0.332599 GB avg = 0.120 KB (by entry num) total key len = 0.077876 GB avg = 0.028 KB total ukey len = 0.055626 GB avg = 0.020 KB total ukey num = 0.002781284 Billion total entry num = 0.002781284 Billion write speed all = 0.007159479 MB/sec (with seq num) write speed all = 0.006771391 MB/sec (without seq num) 2024/11/23-00:47:56.169260 140024953251392 [compaction_job.cc:1948] [default] [JOB 15606] Generated table #237101: 927062 keys, 37326488 bytes, temperature: kWarm 2024/11/23-00:47:56.169323 140024953251392 EVENT_LOG_v1 {"time_micros": 1732294076169290, "cf_name": "default", "job": 15606, "event": "table_file_creation", "file_number": 237101, "file_size": 37326488, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34484352, "index_size": 1752256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25957736, "raw_average_key_size": 28, "raw_value_size": 108260008, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927062, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732294072, "oldest_key_time": 0, "file_creation_time": 1732294072, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 237101, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:47:56.205611 140024953251392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237102.sst) /dev/shm/ToplingTemp/Topling-355990-MSk5at 2024/11/23-00:47:57.040393 140024953251392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237102.sst): first pass time = 0.83's, 160.790'MB/sec 2024/11/23-00:47:58.864210 140024953251392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237102.sst): old prealloc_size = 147639500, real_size = 35391456 2024/11/23-00:47:58.864774 140024953251392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237102.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930381 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1082 GB zip-val = 0.0334 GB avg-val = 116.26 avg-zval = 35.90 2024/11/23-00:47:58.865233 140024953251392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237102.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:47:59.809469 140024953251392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237102.sst): second pass time = 1.82's, 59.341'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 40.099'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3162.547'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.23's, 162.376'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32414.280'MB/sec zip my value time = 1.82's, unzip length = 0.108'GB zip my value throughput = 59.341'MB/sec zip pipeline throughput = 109.614'MB/sec entries = 930381 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.26 avg-zval = 35.90 usrkeys = 930381 avg-key = 20.00 avg-zkey = 1.89 avg-val = 119.40 avg-zval = 35.90 TagRS{ kinds = RS_Key0_TagN bytes = 145480 } TagArray{ size = 416579 bytes = 1510112 } seq expand size = 2916053 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0334 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.5829 value = 3.2385 dict = 1.00 all = 3.5411 } Zip/UnZip{ index = 0.0945 value = 0.3088 dict = 1.00 all = 0.2824 } ---------------------------- total value len = 0.443682 GB avg = 0.120 KB (by entry num) total key len = 0.103927 GB avg = 0.028 KB total ukey len = 0.074233 GB avg = 0.020 KB total ukey num = 0.003711665 Billion total entry num = 0.003711665 Billion write speed all = 0.009550749 MB/sec (with seq num) write speed all = 0.009032873 MB/sec (without seq num) 2024/11/23-00:47:59.848659 140024953251392 [compaction_job.cc:1948] [default] [JOB 15606] Generated table #237102: 930381 keys, 37902665 bytes, temperature: kWarm 2024/11/23-00:47:59.848742 140024953251392 EVENT_LOG_v1 {"time_micros": 1732294079848699, "cf_name": "default", "job": 15606, "event": "table_file_creation", "file_number": 237102, "file_size": 37902665, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3685524867, "table_properties": {"data_size": 33400400, "index_size": 1758272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26050668, "raw_average_key_size": 28, "raw_value_size": 108167182, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930381, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732294076, "oldest_key_time": 0, "file_creation_time": 1732294076, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 237102, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:47:59.914465 140024953251392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237103.sst) /dev/shm/ToplingTemp/Topling-355990-NRtXPv 2024/11/23-00:48:00.825079 140024953251392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237103.sst): first pass time = 0.91's, 147.401'MB/sec 2024/11/23-00:48:02.307914 140024953251392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237103.sst): old prealloc_size = 147639500, real_size = 32960768 2024/11/23-00:48:02.308458 140024953251392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237103.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925976 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 116.95 avg-zval = 33.44 2024/11/23-00:48:02.309237 140024953251392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237103.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:48:03.382559 140024953251392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237103.sst): second pass time = 1.48's, 73.062'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.07's, 34.483'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3379.703'MB/sec (index lex order gen) rebuild zvType time = 0.95's, 0.245'MB/sec write SST data time = 0.13's, 289.269'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34364.943'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.062'MB/sec zip pipeline throughput = 73.062'MB/sec entries = 925976 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.95 avg-zval = 33.44 usrkeys = 925976 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.95 avg-zval = 33.44 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925976 bytes = 3240928 } seq expand size = 6481832 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5114 value = 3.4969 dict = 1.00 all = 3.6220 } Zip/UnZip{ index = 0.0951 value = 0.2860 dict = 1.00 all = 0.2761 } ---------------------------- total value len = 0.558454 GB avg = 0.120 KB (by entry num) total key len = 0.129854 GB avg = 0.028 KB total ukey len = 0.092753 GB avg = 0.020 KB total ukey num = 0.004637641 Billion total entry num = 0.004637641 Billion write speed all = 0.012003919 MB/sec (with seq num) write speed all = 0.011356885 MB/sec (without seq num) 2024/11/23-00:48:03.426052 140024953251392 [compaction_job.cc:1948] [default] [JOB 15606] Generated table #237103: 925976 keys, 37056656 bytes, temperature: kWarm 2024/11/23-00:48:03.426118 140024953251392 EVENT_LOG_v1 {"time_micros": 1732294083426084, "cf_name": "default", "job": 15606, "event": "table_file_creation", "file_number": 237103, "file_size": 37056656, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3511270237, "largest_seqno": 3685525908, "table_properties": {"data_size": 30967216, "index_size": 1761856, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25927328, "raw_average_key_size": 28, "raw_value_size": 108290436, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925976, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732294079, "oldest_key_time": 0, "file_creation_time": 1732294079, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 237103, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:48:03.462401 140024953251392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237104.sst) /dev/shm/ToplingTemp/Topling-355990-yS7W4v 2024/11/23-00:48:04.257904 140024953251392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237104.sst): first pass time = 0.80's, 168.729'MB/sec 2024/11/23-00:48:05.594006 140024953251392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237104.sst): old prealloc_size = 147639500, real_size = 32627344 2024/11/23-00:48:05.594545 140024953251392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237104.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931353 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0306 GB avg-val = 116.11 avg-zval = 32.87 2024/11/23-00:48:05.595361 140024953251392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237104.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:48:06.536335 140024953251392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237104.sst): second pass time = 1.34's, 80.987'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.111'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3452.847'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.287'MB/sec write SST data time = 0.13's, 282.777'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35366.446'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.987'MB/sec zip pipeline throughput = 80.987'MB/sec entries = 931353 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.11 avg-zval = 32.87 usrkeys = 931353 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.11 avg-zval = 32.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931353 bytes = 3376176 } seq expand size = 6519471 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4427 value = 3.5328 dict = 1.00 all = 3.6418 } Zip/UnZip{ index = 0.0958 value = 0.2831 dict = 1.00 all = 0.2746 } ---------------------------- total value len = 0.673114 GB avg = 0.121 KB (by entry num) total key len = 0.155932 GB avg = 0.028 KB total ukey len = 0.111380 GB avg = 0.020 KB total ukey num = 0.005568994 Billion total entry num = 0.005568994 Billion write speed all = 0.014457545 MB/sec (with seq num) write speed all = 0.013680613 MB/sec (without seq num) 2024/11/23-00:48:06.573314 140024953251392 [compaction_job.cc:1948] [default] [JOB 15606] Generated table #237104: 931353 keys, 36854504 bytes, temperature: kWarm 2024/11/23-00:48:06.573378 140024953251392 EVENT_LOG_v1 {"time_micros": 1732294086573346, "cf_name": "default", "job": 15606, "event": "table_file_creation", "file_number": 237104, "file_size": 36854504, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701386, "largest_seqno": 3685525994, "table_properties": {"data_size": 30610560, "index_size": 1783744, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26077884, "raw_average_key_size": 28, "raw_value_size": 108139862, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931353, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732294083, "oldest_key_time": 0, "file_creation_time": 1732294083, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 237104, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:48:06.610550 140024953251392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237105.sst) /dev/shm/ToplingTemp/Topling-355990-NGfUyt 2024/11/23-00:48:07.506115 140024953251392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237105.sst): first pass time = 0.90's, 149.876'MB/sec 2024/11/23-00:48:08.868165 140024953251392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237105.sst): old prealloc_size = 147639500, real_size = 32671424 2024/11/23-00:48:08.868715 140024953251392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237105.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927742 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0307 GB avg-val = 116.67 avg-zval = 33.04 2024/11/23-00:48:08.869468 140024953251392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237105.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:48:09.765530 140024953251392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237105.sst): second pass time = 1.36's, 79.535'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 40.850'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3394.622'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.302'MB/sec write SST data time = 0.13's, 283.259'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35154.805'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.535'MB/sec zip pipeline throughput = 79.535'MB/sec entries = 927742 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.67 avg-zval = 33.04 usrkeys = 927742 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.67 avg-zval = 33.04 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927742 bytes = 3131152 } seq expand size = 6494194 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.3776 value = 3.5314 dict = 1.00 all = 3.6615 } Zip/UnZip{ index = 0.0964 value = 0.2832 dict = 1.00 all = 0.2731 } ---------------------------- total value len = 0.787849 GB avg = 0.121 KB (by entry num) total key len = 0.181909 GB avg = 0.028 KB total ukey len = 0.129935 GB avg = 0.020 KB total ukey num = 0.006496736 Billion total entry num = 0.006496736 Billion write speed all = 0.016910439 MB/sec (with seq num) write speed all = 0.016004128 MB/sec (without seq num) 2024/11/23-00:48:09.799069 140024953251392 [compaction_job.cc:1948] [default] [JOB 15606] Generated table #237105: 927742 keys, 36656736 bytes, temperature: kWarm 2024/11/23-00:48:09.799135 140024953251392 EVENT_LOG_v1 {"time_micros": 1732294089799102, "cf_name": "default", "job": 15606, "event": "table_file_creation", "file_number": 237105, "file_size": 36656736, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 3685525630, "table_properties": {"data_size": 30651312, "index_size": 1787968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25976776, "raw_average_key_size": 28, "raw_value_size": 108240992, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927742, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732294086, "oldest_key_time": 0, "file_creation_time": 1732294086, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 237105, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:48:09.832977 140024953251392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237106.sst) /dev/shm/ToplingTemp/Topling-355990-fLeg3u 2024/11/23-00:48:10.833257 140024953251392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237106.sst): first pass time = 1.00's, 134.185'MB/sec 2024/11/23-00:48:12.200835 140024953251392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237106.sst): old prealloc_size = 147639500, real_size = 33432880 2024/11/23-00:48:12.201386 140024953251392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237106.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926045 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.94 avg-zval = 33.95 2024/11/23-00:48:12.202183 140024953251392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237106.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:48:13.152919 140024953251392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237106.sst): second pass time = 1.37's, 79.220'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.540'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3401.664'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.282'MB/sec write SST data time = 0.13's, 290.501'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33887.875'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.220'MB/sec zip pipeline throughput = 79.220'MB/sec entries = 926045 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.94 avg-zval = 33.95 usrkeys = 926045 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.94 avg-zval = 33.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926045 bytes = 3356928 } seq expand size = 6482315 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5152 value = 3.4443 dict = 1.00 all = 3.5654 } Zip/UnZip{ index = 0.0951 value = 0.2903 dict = 1.00 all = 0.2805 } ---------------------------- total value len = 0.902620 GB avg = 0.122 KB (by entry num) total key len = 0.207838 GB avg = 0.028 KB total ukey len = 0.148456 GB avg = 0.020 KB total ukey num = 0.007422781 Billion total entry num = 0.007422781 Billion write speed all = 0.019362796 MB/sec (with seq num) write speed all = 0.018327361 MB/sec (without seq num) 2024/11/23-00:48:13.184941 140024953251392 [compaction_job.cc:1948] [default] [JOB 15606] Generated table #237106: 926045 keys, 37644962 bytes, temperature: kWarm 2024/11/23-00:48:13.185004 140024953251392 EVENT_LOG_v1 {"time_micros": 1732294093184972, "cf_name": "default", "job": 15606, "event": "table_file_creation", "file_number": 237106, "file_size": 37644962, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3685522645, "table_properties": {"data_size": 31439840, "index_size": 1761344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25929260, "raw_average_key_size": 28, "raw_value_size": 108288510, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926045, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732294089, "oldest_key_time": 0, "file_creation_time": 1732294089, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 237106, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:48:13.220927 140024953251392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237107.sst) /dev/shm/ToplingTemp/Topling-355990-OhuGav 2024/11/23-00:48:14.041629 140024953251392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237107.sst): first pass time = 0.82's, 163.548'MB/sec 2024/11/23-00:48:15.351693 140024953251392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237107.sst): old prealloc_size = 147639500, real_size = 33496288 2024/11/23-00:48:15.352292 140024953251392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237107.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926239 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 116.91 avg-zval = 34.01 2024/11/23-00:48:15.353056 140024953251392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237107.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:48:16.299930 140024953251392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237107.sst): second pass time = 1.31's, 82.699'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.523'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3050.700'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.283'MB/sec write SST data time = 0.13's, 290.388'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33498.295'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.699'MB/sec zip pipeline throughput = 82.699'MB/sec entries = 926239 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.91 avg-zval = 34.01 usrkeys = 926239 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.91 avg-zval = 34.01 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926239 bytes = 3126064 } seq expand size = 6483673 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5124 value = 3.4373 dict = 1.00 all = 3.5813 } Zip/UnZip{ index = 0.0951 value = 0.2909 dict = 1.00 all = 0.2792 } ---------------------------- total value len = 1.017386 GB avg = 0.122 KB (by entry num) total key len = 0.233773 GB avg = 0.028 KB total ukey len = 0.166980 GB avg = 0.020 KB total ukey num = 0.008349020 Billion total entry num = 0.008349020 Billion write speed all = 0.021814978 MB/sec (with seq num) write speed all = 0.020650402 MB/sec (without seq num) 2024/11/23-00:48:16.332872 140024953251392 [compaction_job.cc:1948] [default] [JOB 15606] Generated table #237107: 926239 keys, 37477014 bytes, temperature: kWarm 2024/11/23-00:48:16.332935 140024953251392 EVENT_LOG_v1 {"time_micros": 1732294096332903, "cf_name": "default", "job": 15606, "event": "table_file_creation", "file_number": 237107, "file_size": 37477014, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3563652023, "largest_seqno": 3685525474, "table_properties": {"data_size": 31502352, "index_size": 1762176, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25934692, "raw_average_key_size": 28, "raw_value_size": 108283106, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926239, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732294093, "oldest_key_time": 0, "file_creation_time": 1732294093, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 237107, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:48:16.369590 140024953251392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237108.sst) /dev/shm/ToplingTemp/Topling-355990-uhKY5u 2024/11/23-00:48:17.271380 140024953251392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237108.sst): first pass time = 0.90's, 148.841'MB/sec 2024/11/23-00:48:18.588490 140024953251392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237108.sst): old prealloc_size = 147639500, real_size = 33244624 2024/11/23-00:48:18.589046 140024953251392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237108.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927339 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 116.73 avg-zval = 33.70 2024/11/23-00:48:18.589837 140024953251392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237108.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:48:19.541127 140024953251392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237108.sst): second pass time = 1.32's, 82.230'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.202'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3289.735'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.281'MB/sec write SST data time = 0.13's, 292.136'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33685.310'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.230'MB/sec zip pipeline throughput = 82.230'MB/sec entries = 927339 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.73 avg-zval = 33.70 usrkeys = 927339 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.73 avg-zval = 33.70 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927339 bytes = 3245696 } seq expand size = 6491373 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5165 value = 3.4642 dict = 1.00 all = 3.5940 } Zip/UnZip{ index = 0.0951 value = 0.2887 dict = 1.00 all = 0.2782 } ---------------------------- total value len = 1.132130 GB avg = 0.122 KB (by entry num) total key len = 0.259738 GB avg = 0.028 KB total ukey len = 0.185527 GB avg = 0.020 KB total ukey num = 0.009276359 Billion total entry num = 0.009276359 Billion write speed all = 0.024266987 MB/sec (with seq num) write speed all = 0.022973133 MB/sec (without seq num) 2024/11/23-00:48:19.580430 140024953251392 [compaction_job.cc:1948] [default] [JOB 15606] Generated table #237108: 927339 keys, 37344826 bytes, temperature: kWarm 2024/11/23-00:48:19.580495 140024953251392 EVENT_LOG_v1 {"time_micros": 1732294099580463, "cf_name": "default", "job": 15606, "event": "table_file_creation", "file_number": 237108, "file_size": 37344826, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3504428200, "largest_seqno": 3685524959, "table_properties": {"data_size": 31249024, "index_size": 1763584, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25965492, "raw_average_key_size": 28, "raw_value_size": 108252366, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927339, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732294096, "oldest_key_time": 0, "file_creation_time": 1732294096, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 237108, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:48:19.617058 140024953251392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237109.sst) /dev/shm/ToplingTemp/Topling-355990-vlZKXu 2024/11/23-00:48:20.517233 140024953251392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237109.sst): first pass time = 0.90's, 149.108'MB/sec 2024/11/23-00:48:21.841253 140024953251392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237109.sst): old prealloc_size = 147639500, real_size = 32025072 2024/11/23-00:48:21.841862 140024953251392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237109.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931571 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0300 GB avg-val = 116.08 avg-zval = 32.23 2024/11/23-00:48:21.843284 140024953251392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237109.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:48:22.758257 140024953251392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237109.sst): second pass time = 1.32's, 81.711'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 39.281'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3011.148'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.295'MB/sec write SST data time = 0.13's, 280.871'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 27875.705'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.711'MB/sec zip pipeline throughput = 81.711'MB/sec entries = 931571 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.08 avg-zval = 32.23 usrkeys = 931571 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.08 avg-zval = 32.23 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931571 bytes = 3144064 } seq expand size = 6520997 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.5164 value = 3.6020 dict = 1.00 all = 3.7262 } Zip/UnZip{ index = 0.0951 value = 0.2776 dict = 1.00 all = 0.2684 } ---------------------------- total value len = 1.246785 GB avg = 0.122 KB (by entry num) total key len = 0.285822 GB avg = 0.028 KB total ukey len = 0.204159 GB avg = 0.020 KB total ukey num = 0.010207930 Billion total entry num = 0.010207930 Billion write speed all = 0.026719247 MB/sec (with seq num) write speed all = 0.025295539 MB/sec (without seq num) 2024/11/23-00:48:22.789004 140024953251392 [compaction_job.cc:1948] [default] [JOB 15606] Generated table #237109: 931571 keys, 36019938 bytes, temperature: kWarm 2024/11/23-00:48:22.789068 140024953251392 EVENT_LOG_v1 {"time_micros": 1732294102789036, "cf_name": "default", "job": 15606, "event": "table_file_creation", "file_number": 237109, "file_size": 36019938, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3587898319, "largest_seqno": 3685525325, "table_properties": {"data_size": 30020320, "index_size": 1771648, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26083988, "raw_average_key_size": 28, "raw_value_size": 108133874, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931571, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732294099, "oldest_key_time": 0, "file_creation_time": 1732294099, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 237109, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:48:22.822693 140024953251392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237110.sst) /dev/shm/ToplingTemp/Topling-355990-iMGrNv 2024/11/23-00:48:23.727302 140024953251392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237110.sst): first pass time = 0.90's, 148.379'MB/sec 2024/11/23-00:48:25.094217 140024953251392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237110.sst): old prealloc_size = 147639500, real_size = 35073216 2024/11/23-00:48:25.094867 140024953251392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237110.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1081809 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1039 GB zip-val = 0.0327 GB avg-val = 96.07 avg-zval = 30.24 2024/11/23-00:48:25.095021 140024953251392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237110.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:48:25.720465 140024953251392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237110.sst): second pass time = 1.37's, 76.067'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.63's, 57.571'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3319.098'MB/sec (index lex order gen) rebuild zvType time = 0.49's, 0.555'MB/sec write SST data time = 0.14's, 261.489'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.029'MB, throughput = 30972.352'MB/sec zip my value time = 1.37's, unzip length = 0.104'GB zip my value throughput = 76.067'MB/sec zip pipeline throughput = 76.067'MB/sec entries = 1081809 avg-key = 28.00 avg-zkey = 1.93 avg-val = 96.07 avg-zval = 30.24 usrkeys = 1081809 avg-key = 20.00 avg-zkey = 1.93 avg-val = 96.16 avg-zval = 30.24 TagRS{ kinds = RS_Key0_TagN bytes = 169120 } TagArray{ size = 14639 bytes = 40272 } seq expand size = 102473 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0216 GB value = 0.1039 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0327 GB dict = 1.03 MB all = 0.0361 GB } UnZip/Zip{ index = 10.3797 value = 3.1764 dict = 1.00 all = 3.7228 } Zip/UnZip{ index = 0.0963 value = 0.3148 dict = 1.00 all = 0.2686 } ---------------------------- total value len = 1.350815 GB avg = 0.120 KB (by entry num) total key len = 0.316113 GB avg = 0.028 KB total ukey len = 0.225795 GB avg = 0.020 KB total ukey num = 0.011289739 Billion total entry num = 0.011289739 Billion write speed all = 0.029059467 MB/sec (with seq num) write speed all = 0.027484959 MB/sec (without seq num) 2024/11/23-00:48:25.753525 140024953251392 [compaction_job.cc:1948] [default] [JOB 15606] Generated table #237110: 1081809 keys, 36052737 bytes, temperature: kWarm 2024/11/23-00:48:25.753607 140024953251392 EVENT_LOG_v1 {"time_micros": 1732294105753575, "cf_name": "default", "job": 15606, "event": "table_file_creation", "file_number": 237110, "file_size": 36052737, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3685523657, "table_properties": {"data_size": 32718128, "index_size": 2084480, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30290652, "raw_average_key_size": 28, "raw_value_size": 103927183, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1081809, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732294102, "oldest_key_time": 0, "file_creation_time": 1732294102, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 237110, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:48:25.787522 140024953251392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237111.sst) /dev/shm/ToplingTemp/Topling-355990-0UUWrw 2024/11/23-00:48:26.584318 140024953251392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237111.sst): first pass time = 0.80's, 168.455'MB/sec 2024/11/23-00:48:27.977675 140024953251392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237111.sst): old prealloc_size = 147639500, real_size = 34573136 2024/11/23-00:48:27.978254 140024953251392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237111.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922013 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0326 GB avg-val = 117.57 avg-zval = 35.34 2024/11/23-00:48:27.978828 140024953251392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237111.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:48:28.831373 140024953251392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237111.sst): second pass time = 1.39's, 77.864'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.140'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3148.334'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.319'MB/sec write SST data time = 0.13's, 290.249'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 20324.533'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 77.864'MB/sec zip pipeline throughput = 77.864'MB/sec entries = 922013 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.57 avg-zval = 35.34 usrkeys = 922013 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.13 avg-zval = 35.34 TagRS{ kinds = RS_Key0_TagN bytes = 144160 } TagArray{ size = 600780 bytes = 2102752 } seq expand size = 4205460 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0326 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5134 value = 3.3264 dict = 1.00 all = 3.5618 } Zip/UnZip{ index = 0.0951 value = 0.3006 dict = 1.00 all = 0.2808 } ---------------------------- total value len = 1.463421 GB avg = 0.120 KB (by entry num) total key len = 0.341929 GB avg = 0.028 KB total ukey len = 0.244235 GB avg = 0.020 KB total ukey num = 0.012211752 Billion total entry num = 0.012211752 Billion write speed all = 0.031470886 MB/sec (with seq num) write speed all = 0.029767883 MB/sec (without seq num) 2024/11/23-00:48:28.867485 140024953251392 [compaction_job.cc:1948] [default] [JOB 15606] Generated table #237111: 922013 keys, 37682060 bytes, temperature: kWarm 2024/11/23-00:48:28.867547 140024953251392 EVENT_LOG_v1 {"time_micros": 1732294108867514, "cf_name": "default", "job": 15606, "event": "table_file_creation", "file_number": 237111, "file_size": 37682060, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3496930348, "table_properties": {"data_size": 32588480, "index_size": 1753984, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25816364, "raw_average_key_size": 28, "raw_value_size": 108401374, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922013, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732294105, "oldest_key_time": 0, "file_creation_time": 1732294105, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 237111, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:48:28.904306 140024953251392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237112.sst) /dev/shm/ToplingTemp/Topling-355990-kRnf9u 2024/11/23-00:48:29.827730 140024953251392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237112.sst): first pass time = 0.92's, 145.355'MB/sec 2024/11/23-00:48:31.139436 140024953251392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237112.sst): old prealloc_size = 147639500, real_size = 30903232 2024/11/23-00:48:31.140002 140024953251392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237112.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925214 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0289 GB avg-val = 117.07 avg-zval = 31.24 2024/11/23-00:48:31.140786 140024953251392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237112.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:48:31.992107 140024953251392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237112.sst): second pass time = 1.31's, 82.615'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 41.043'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3261.473'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 271.670'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33200.565'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.615'MB/sec zip pipeline throughput = 82.615'MB/sec entries = 925214 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.07 avg-zval = 31.24 usrkeys = 925214 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.07 avg-zval = 31.24 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925214 bytes = 3238256 } seq expand size = 6476498 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.4428 value = 3.7478 dict = 1.00 all = 3.8353 } Zip/UnZip{ index = 0.0958 value = 0.2668 dict = 1.00 all = 0.2607 } ---------------------------- total value len = 1.578210 GB avg = 0.120 KB (by entry num) total key len = 0.367835 GB avg = 0.028 KB total ukey len = 0.262739 GB avg = 0.020 KB total ukey num = 0.013136966 Billion total entry num = 0.013136966 Billion write speed all = 0.033921602 MB/sec (with seq num) write speed all = 0.032089673 MB/sec (without seq num) 2024/11/23-00:48:32.022947 140024953251392 [compaction_job.cc:1948] [default] [JOB 15606] Generated table #237112: 925214 keys, 34995072 bytes, temperature: kWarm 2024/11/23-00:48:32.023010 140024953251392 EVENT_LOG_v1 {"time_micros": 1732294112022978, "cf_name": "default", "job": 15606, "event": "table_file_creation", "file_number": 237112, "file_size": 34995072, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3496930349, "largest_seqno": 3685524615, "table_properties": {"data_size": 28899760, "index_size": 1771968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25905992, "raw_average_key_size": 28, "raw_value_size": 108311844, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925214, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732294108, "oldest_key_time": 0, "file_creation_time": 1732294108, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 237112, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:48:32.058404 140024953251392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237113.sst) /dev/shm/ToplingTemp/Topling-355990-v15Wbw 2024/11/23-00:48:32.842985 140024953251392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237113.sst): first pass time = 0.78's, 171.078'MB/sec 2024/11/23-00:48:34.204693 140024953251392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237113.sst): old prealloc_size = 147639500, real_size = 33377824 2024/11/23-00:48:34.205237 140024953251392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237113.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927190 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.76 avg-zval = 33.85 2024/11/23-00:48:34.206036 140024953251392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237113.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:48:35.129949 140024953251392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237113.sst): second pass time = 1.36's, 79.545'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.631'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3376.131'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.291'MB/sec write SST data time = 0.13's, 290.829'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33279.704'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.545'MB/sec zip pipeline throughput = 79.545'MB/sec entries = 927190 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.76 avg-zval = 33.85 usrkeys = 927190 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.76 avg-zval = 33.85 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927190 bytes = 3361072 } seq expand size = 6490330 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5122 value = 3.4497 dict = 1.00 all = 3.5702 } Zip/UnZip{ index = 0.0951 value = 0.2899 dict = 1.00 all = 0.2801 } ---------------------------- total value len = 1.692957 GB avg = 0.120 KB (by entry num) total key len = 0.393796 GB avg = 0.028 KB total ukey len = 0.281283 GB avg = 0.020 KB total ukey num = 0.014064156 Billion total entry num = 0.014064156 Billion write speed all = 0.036372301 MB/sec (with seq num) write speed all = 0.034411185 MB/sec (without seq num) 2024/11/23-00:48:35.165135 140024953251392 [compaction_job.cc:1948] [default] [JOB 15606] Generated table #237113: 927190 keys, 37593504 bytes, temperature: kWarm 2024/11/23-00:48:35.165202 140024953251392 EVENT_LOG_v1 {"time_micros": 1732294115165166, "cf_name": "default", "job": 15606, "event": "table_file_creation", "file_number": 237113, "file_size": 37593504, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3375334738, "largest_seqno": 3685525672, "table_properties": {"data_size": 31381840, "index_size": 1764032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25961320, "raw_average_key_size": 28, "raw_value_size": 108256424, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927190, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732294112, "oldest_key_time": 0, "file_creation_time": 1732294112, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 237113, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:48:35.199882 140024953251392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237114.sst) /dev/shm/ToplingTemp/Topling-355990-0iZkAs 2024/11/23-00:48:36.106353 140024953251392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237114.sst): first pass time = 0.91's, 148.073'MB/sec 2024/11/23-00:48:37.438854 140024953251392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237114.sst): old prealloc_size = 147639500, real_size = 32365136 2024/11/23-00:48:37.439375 140024953251392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237114.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926212 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0304 GB avg-val = 116.91 avg-zval = 32.79 2024/11/23-00:48:37.441942 140024953251392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237114.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:48:38.331818 140024953251392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237114.sst): second pass time = 1.33's, 81.331'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 40.963'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3521.665'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.304'MB/sec write SST data time = 0.13's, 278.930'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 26113.794'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.331'MB/sec zip pipeline throughput = 81.331'MB/sec entries = 926212 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.91 avg-zval = 32.79 usrkeys = 926212 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.91 avg-zval = 32.79 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926212 bytes = 3357536 } seq expand size = 6483484 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5133 value = 3.5653 dict = 1.00 all = 3.6694 } Zip/UnZip{ index = 0.0951 value = 0.2805 dict = 1.00 all = 0.2725 } ---------------------------- total value len = 1.807724 GB avg = 0.121 KB (by entry num) total key len = 0.419730 GB avg = 0.028 KB total ukey len = 0.299807 GB avg = 0.020 KB total ukey num = 0.014990368 Billion total entry num = 0.014990368 Billion write speed all = 0.038822572 MB/sec (with seq num) write speed all = 0.036732421 MB/sec (without seq num) 2024/11/23-00:48:38.362880 140024953251392 [compaction_job.cc:1948] [default] [JOB 15606] Generated table #237114: 926212 keys, 36577318 bytes, temperature: kWarm 2024/11/23-00:48:38.362943 140024953251392 EVENT_LOG_v1 {"time_micros": 1732294118362911, "cf_name": "default", "job": 15606, "event": "table_file_creation", "file_number": 237114, "file_size": 36577318, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3685525867, "table_properties": {"data_size": 30371392, "index_size": 1761984, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25933936, "raw_average_key_size": 28, "raw_value_size": 108283916, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926212, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732294115, "oldest_key_time": 0, "file_creation_time": 1732294115, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 237114, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:48:38.403561 140024953251392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237115.sst) /dev/shm/ToplingTemp/Topling-355990-a9wb8u 2024/11/23-00:48:38.583134 140024953251392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237115.sst): first pass time = 0.18's, 236.015'MB/sec 2024/11/23-00:48:38.989895 140024953251392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237115.sst): old prealloc_size = 147639500, real_size = 12366208 2024/11/23-00:48:38.990119 140024953251392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237115.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 291573 prefix = 4 raw-key = 0.0047 GB zip-key = 0.0006 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0342 GB zip-val = 0.0117 GB avg-val = 117.32 avg-zval = 40.23 2024/11/23-00:48:38.990399 140024953251392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237115.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0007 GB, waited 0.000 sec, Key+Value = 0.042 GB 2024/11/23-00:48:39.251585 140024953251392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00/237115.sst): second pass time = 0.41's, 84.224'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.26's, 52.354'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2836.328'MB/sec (index lex order gen) rebuild zvType time = 0.22's, 0.331'MB/sec write SST data time = 0.04's, 334.326'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.347'MB, throughput = 11041.159'MB/sec zip my value time = 0.41's, unzip length = 0.034'GB zip my value throughput = 84.224'MB/sec zip pipeline throughput = 84.224'MB/sec entries = 291573 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.32 avg-zval = 40.23 usrkeys = 291573 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.32 avg-zval = 40.23 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 291573 bytes = 1056960 } seq expand size = 2041011 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0058 GB value = 0.0342 GB dict = 0.35 MB all = 0.0424 GB } __ZipSize{ index = 0.0006 GB value = 0.0117 GB dict = 0.35 MB all = 0.0137 GB } UnZip/Zip{ index = 10.3553 value = 2.9163 dict = 1.00 all = 3.0929 } Zip/UnZip{ index = 0.0966 value = 0.3429 dict = 1.00 all = 0.3233 } ---------------------------- total value len = 1.843973 GB avg = 0.121 KB (by entry num) total key len = 0.427894 GB avg = 0.028 KB total ukey len = 0.305639 GB avg = 0.020 KB total ukey num = 0.015281941 Billion total entry num = 0.015281941 Billion write speed all = 0.039596022 MB/sec (with seq num) write speed all = 0.037465249 MB/sec (without seq num) 2024/11/23-00:48:39.267250 140024953251392 [compaction_job.cc:1948] [default] [JOB 15606] Generated table #237115: 291573 keys, 13699769 bytes, temperature: kWarm 2024/11/23-00:48:39.267309 140024953251392 EVENT_LOG_v1 {"time_micros": 1732294119267279, "cf_name": "default", "job": 15606, "event": "table_file_creation", "file_number": 237115, "file_size": 13699769, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369194706, "largest_seqno": 3685313256, "table_properties": {"data_size": 11729968, "index_size": 563136, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 8164044, "raw_average_key_size": 28, "raw_value_size": 34208206, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 291573, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732294118, "oldest_key_time": 0, "file_creation_time": 1732294118, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 237115, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:48:39.280291 140024953251392 [compaction_job.cc:703] job-15606: subcompact[0], size: 0.608245 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15606/att-00 [237099,237100,237101,237102,237103,237104,237105,237106,237107,237108,237109,237110,237111,237112,237113,237114,237115] 2024/11/23-00:48:39.389930 140024953251392 (Original Log Time 2024/11/23-00:47:43.219951) EVENT_LOG_v1 {"time_micros": 1732294063219912, "job": 15606, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [237095], "files_L3": [237063, 237064, 237065, 237066, 237067, 237068, 237069, 237070, 237026, 237027, 237028, 237029, 237030, 237031, 237032, 237033, 237046], "score": 0.999994, "input_data_size": 630667814, "oldest_snapshot_seqno": 3685544965} 2024/11/23-00:48:39.389934 140024953251392 (Original Log Time 2024/11/23-00:48:39.389079) [compaction_job.cc:2022] [default] [JOB 15606] Compacted 1@2 + 17@3 files to L3 => 608245416 bytes 2024/11/23-00:48:39.389935 140024953251392 (Original Log Time 2024/11/23-00:48:39.389862) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 17 0 0 0] max score 0.02, MB/sec: 11.2 rd, 10.8 wr, level 3, files in(1, 17) out(17 +0 blob) MB in(27.1, 574.3 +0.0 blob) out(580.1 +0.0 blob), read-write-amplify(43.6) write-amplify(21.4) OK, records in: 15387254, records dropped: 105313 output_compression: Snappy 2024/11/23-00:48:39.389941 140024953251392 (Original Log Time 2024/11/23-00:48:39.389910) EVENT_LOG_v1 {"time_micros": 1732294119389886, "cf": "default", "job": 15606, "event": "compaction_finished", "compaction_time_micros": 56060349, "compaction_time_cpu_micros": 24365140, "output_level": 3, "num_output_files": 17, "total_output_size": 608245416, "num_input_records": 15387254, "num_output_records": 15281941, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 414669396, "file_range_sync_nanos": 0, "file_fsync_nanos": 16077, "file_prepare_write_nanos": 41704, "lsm_state": [0, 0, 0, 17, 0, 0, 0]} 2024/11/23-00:48:39.518509 140024953251392 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732294063,"executesMs":56293,"compactionJobId":15606,"attempt":0,"compactionInputRawBytes":2204811581,"compactionInputZipBytes":630667814,"compactionOutputRawBytes":0,"compactionOutputZipBytes":608245416,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-46ad6b47b84245e78c382d75516d29b2-d88762b4b121af42-00"}}, response = 2024/11/23-00:48:39.518587 140024953251392 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-15606/att-00: olev 3, work 56.293 s, result 4.902 ms, install 0.881 ms, input{raw 2.053 GiB zip 601.452 MiB}