2024/11/22-16:21:01.269095 140025179854400 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00: fork to child time = 0.031451 sec 2024/11/22-16:21:01.270097 140025179854400 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00 2024/11/22-16:21:01.278349 140025179854400 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00 2024/11/22-16:21:01.279743 140025179854400 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-16:21:01.280053 140025179854400 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-16:21:01.332667 140025179854400 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-04363/att-00/MANIFEST-206347 2024/11/22-16:21:01.349150 140025179854400 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-04363/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 214165, last_sequence is 3374158269, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 213989 2024/11/22-16:21:01.349158 140025179854400 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-16:21:01.349284 140025179854400 [version_set.cc:5721] Creating manifest 214165 2024/11/22-16:21:01.379964 140025179854400 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-16:21:01.399504 140025179854400 [compaction_job.cc:2437] [default] [JOB 4363] Compacting 1@2 + 14@3 files to L3, score 0.88, subcompactions 1 : 1 2024/11/22-16:21:01.399523 140025179854400 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 3404 Base level 2, inputs: [214117(27MB)], [214065(36MB) 214066(36MB) 214067(36MB) 214068(35MB) 214069(36MB) 214070(37MB) 214071(38MB) 214072(38MB) 214073(36MB) 214074(35MB) 214075(36MB) 214076(36MB) 214082(36MB) 214156(36MB)] 2024/11/22-16:21:01.407271 140025179854400 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-16:21:01.407311 140025179854400 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-16:21:01.407337 140025179854400 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-16:21:01.407345 140025179854400 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-16:21:01.407360 140025179854400 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-16:21:01.875195 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214166.sst) /dev/shm/ToplingTemp/Topling-195935-E3IKrT 2024/11/22-16:21:02.419374 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214166.sst): first pass time = 0.54's, 246.660'MB/sec 2024/11/22-16:21:03.991972 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214166.sst): old prealloc_size = 147639500, real_size = 33866816 2024/11/22-16:21:03.992530 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214166.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921091 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0319 GB avg-val = 117.72 avg-zval = 34.60 2024/11/22-16:21:03.993308 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214166.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:21:04.920196 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214166.sst): second pass time = 1.57's, 68.979'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.761'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3281.203'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.288'MB/sec write SST data time = 0.13's, 294.893'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 35882.419'MB/sec zip my value time = 1.57's, unzip length = 0.108'GB zip my value throughput = 68.979'MB/sec zip pipeline throughput = 135.224'MB/sec entries = 921091 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.72 avg-zval = 34.60 usrkeys = 921091 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.72 avg-zval = 34.60 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921091 bytes = 3108704 } seq expand size = 6447637 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.4442 value = 3.4019 dict = 1.00 all = 3.5474 } Zip/UnZip{ index = 0.0957 value = 0.2940 dict = 1.00 all = 0.2819 } ---------------------------- total value len = 0.114875 GB avg = 0.125 KB (by entry num) total key len = 0.025791 GB avg = 0.028 KB total ukey len = 0.018422 GB avg = 0.020 KB total ukey num = 0.000921091 Billion total entry num = 0.000921091 Billion write speed all = 0.005224960 MB/sec (with seq num) write speed all = 0.004951252 MB/sec (without seq num) 2024/11/22-16:21:04.978774 140025179854400 [compaction_job.cc:1948] [default] [JOB 4363] Generated table #214166: 921091 keys, 37835028 bytes, temperature: kWarm 2024/11/22-16:21:04.978876 140025179854400 EVENT_LOG_v1 {"time_micros": 1732263664978813, "cf_name": "default", "job": 4363, "event": "table_file_creation", "file_number": 214166, "file_size": 37835028, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702550, "largest_seqno": 3373735469, "table_properties": {"data_size": 31872496, "index_size": 1763840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25790548, "raw_average_key_size": 28, "raw_value_size": 108427270, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921091, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263661, "oldest_key_time": 0, "file_creation_time": 1732263661, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 214166, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:21:05.017788 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214167.sst) /dev/shm/ToplingTemp/Topling-195935-lLmAFS 2024/11/22-16:21:05.833842 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214167.sst): first pass time = 0.82's, 164.478'MB/sec 2024/11/22-16:21:07.417290 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214167.sst): old prealloc_size = 147639500, real_size = 34528064 2024/11/22-16:21:07.417875 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214167.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922943 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0325 GB avg-val = 117.42 avg-zval = 35.25 2024/11/22-16:21:07.418615 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214167.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:21:08.312719 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214167.sst): second pass time = 1.58's, 68.476'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 42.997'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3115.918'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.300'MB/sec write SST data time = 0.12's, 309.249'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 29227.073'MB/sec zip my value time = 1.58's, unzip length = 0.108'GB zip my value throughput = 68.476'MB/sec zip pipeline throughput = 122.615'MB/sec entries = 922943 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.42 avg-zval = 35.25 usrkeys = 922943 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.42 avg-zval = 35.25 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922943 bytes = 3114944 } seq expand size = 6460601 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.4462 value = 3.3315 dict = 1.00 all = 3.4861 } Zip/UnZip{ index = 0.0957 value = 0.3002 dict = 1.00 all = 0.2869 } ---------------------------- total value len = 0.229711 GB avg = 0.125 KB (by entry num) total key len = 0.051633 GB avg = 0.028 KB total ukey len = 0.036881 GB avg = 0.020 KB total ukey num = 0.001844034 Billion total entry num = 0.001844034 Billion write speed all = 0.010449086 MB/sec (with seq num) write speed all = 0.009901188 MB/sec (without seq num) 2024/11/22-16:21:08.347197 140025179854400 [compaction_job.cc:1948] [default] [JOB 4363] Generated table #214167: 922943 keys, 38500728 bytes, temperature: kWarm 2024/11/22-16:21:08.347254 140025179854400 EVENT_LOG_v1 {"time_micros": 1732263668347224, "cf_name": "default", "job": 4363, "event": "table_file_creation", "file_number": 214167, "file_size": 38500728, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702561, "largest_seqno": 3373736086, "table_properties": {"data_size": 32530096, "index_size": 1767040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25842404, "raw_average_key_size": 28, "raw_value_size": 108375446, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922943, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263665, "oldest_key_time": 0, "file_creation_time": 1732263665, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 214167, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:21:08.382809 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214168.sst) /dev/shm/ToplingTemp/Topling-195935-izwLsT 2024/11/22-16:21:09.215235 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214168.sst): first pass time = 0.83's, 161.244'MB/sec 2024/11/22-16:21:10.824117 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214168.sst): old prealloc_size = 147639500, real_size = 34664832 2024/11/22-16:21:10.824653 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214168.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922547 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0327 GB avg-val = 117.49 avg-zval = 35.42 2024/11/22-16:21:10.825410 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214168.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:21:11.731579 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214168.sst): second pass time = 1.61's, 67.394'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 42.578'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3398.988'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.297'MB/sec write SST data time = 0.13's, 298.957'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 31757.629'MB/sec zip my value time = 1.61's, unzip length = 0.108'GB zip my value throughput = 67.394'MB/sec zip pipeline throughput = 133.708'MB/sec entries = 922547 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.49 avg-zval = 35.42 usrkeys = 922547 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.49 avg-zval = 35.42 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922547 bytes = 3113616 } seq expand size = 6457829 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0327 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.5122 value = 3.3167 dict = 1.00 all = 3.4738 } Zip/UnZip{ index = 0.0951 value = 0.3015 dict = 1.00 all = 0.2879 } ---------------------------- total value len = 0.344555 GB avg = 0.125 KB (by entry num) total key len = 0.077464 GB avg = 0.028 KB total ukey len = 0.055332 GB avg = 0.020 KB total ukey num = 0.002766581 Billion total entry num = 0.002766581 Billion write speed all = 0.015671773 MB/sec (with seq num) write speed all = 0.014849873 MB/sec (without seq num) 2024/11/22-16:21:11.791663 140025179854400 [compaction_job.cc:1948] [default] [JOB 4363] Generated table #214168: 922547 keys, 38637028 bytes, temperature: kWarm 2024/11/22-16:21:11.791712 140025179854400 EVENT_LOG_v1 {"time_micros": 1732263671791684, "cf_name": "default", "job": 4363, "event": "table_file_creation", "file_number": 214168, "file_size": 38637028, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702585, "largest_seqno": 3373735517, "table_properties": {"data_size": 32678832, "index_size": 1755200, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25831316, "raw_average_key_size": 28, "raw_value_size": 108386418, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922547, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263668, "oldest_key_time": 0, "file_creation_time": 1732263668, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 214168, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:21:11.850424 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214169.sst) /dev/shm/ToplingTemp/Topling-195935-f15FNU 2024/11/22-16:21:12.645018 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214169.sst): first pass time = 0.79's, 168.922'MB/sec 2024/11/22-16:21:14.191768 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214169.sst): old prealloc_size = 147639500, real_size = 34011920 2024/11/22-16:21:14.192333 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214169.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921047 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0320 GB avg-val = 117.72 avg-zval = 34.77 2024/11/22-16:21:14.193097 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214169.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:21:15.115230 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214169.sst): second pass time = 1.55's, 70.147'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 41.128'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3255.981'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.289'MB/sec write SST data time = 0.13's, 300.608'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 33223.213'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 70.147'MB/sec zip pipeline throughput = 121.951'MB/sec entries = 921047 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.72 avg-zval = 34.77 usrkeys = 921047 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.72 avg-zval = 34.77 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921047 bytes = 3108544 } seq expand size = 6447329 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.5123 value = 3.3853 dict = 1.00 all = 3.5339 } Zip/UnZip{ index = 0.0951 value = 0.2954 dict = 1.00 all = 0.2830 } ---------------------------- total value len = 0.459431 GB avg = 0.125 KB (by entry num) total key len = 0.103254 GB avg = 0.028 KB total ukey len = 0.073753 GB avg = 0.020 KB total ukey num = 0.003687628 Billion total entry num = 0.003687628 Billion write speed all = 0.020892775 MB/sec (with seq num) write speed all = 0.019797386 MB/sec (without seq num) 2024/11/22-16:21:15.152198 140025179854400 [compaction_job.cc:1948] [default] [JOB 4363] Generated table #214169: 921047 keys, 37979788 bytes, temperature: kWarm 2024/11/22-16:21:15.152253 140025179854400 EVENT_LOG_v1 {"time_micros": 1732263675152223, "cf_name": "default", "job": 4363, "event": "table_file_creation", "file_number": 214169, "file_size": 37979788, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702617, "largest_seqno": 3373735548, "table_properties": {"data_size": 32029184, "index_size": 1752320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25789316, "raw_average_key_size": 28, "raw_value_size": 108428502, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921047, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263671, "oldest_key_time": 0, "file_creation_time": 1732263671, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 214169, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:21:15.186309 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214170.sst) /dev/shm/ToplingTemp/Topling-195935-9AMFSU 2024/11/22-16:21:16.036346 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214170.sst): first pass time = 0.85's, 157.903'MB/sec 2024/11/22-16:21:17.652389 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214170.sst): old prealloc_size = 147639500, real_size = 34396400 2024/11/22-16:21:17.652942 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214170.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922746 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0324 GB avg-val = 117.45 avg-zval = 35.11 2024/11/22-16:21:17.653693 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214170.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:21:18.557970 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214170.sst): second pass time = 1.62's, 67.094'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 42.370'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3315.254'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.296'MB/sec write SST data time = 0.13's, 303.559'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33516.061'MB/sec zip my value time = 1.62's, unzip length = 0.108'GB zip my value throughput = 67.094'MB/sec zip pipeline throughput = 99.863'MB/sec entries = 922746 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.45 avg-zval = 35.11 usrkeys = 922746 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.45 avg-zval = 35.11 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922746 bytes = 3114288 } seq expand size = 6459222 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0324 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.4413 value = 3.3453 dict = 1.00 all = 3.4981 } Zip/UnZip{ index = 0.0958 value = 0.2989 dict = 1.00 all = 0.2859 } ---------------------------- total value len = 0.574271 GB avg = 0.125 KB (by entry num) total key len = 0.129090 GB avg = 0.028 KB total ukey len = 0.092207 GB avg = 0.020 KB total ukey num = 0.004610374 Billion total entry num = 0.004610374 Billion write speed all = 0.026112847 MB/sec (with seq num) write speed all = 0.024743537 MB/sec (without seq num) 2024/11/22-16:21:18.590853 140025179854400 [compaction_job.cc:1948] [default] [JOB 4363] Generated table #214170: 922746 keys, 38368878 bytes, temperature: kWarm 2024/11/22-16:21:18.590911 140025179854400 EVENT_LOG_v1 {"time_micros": 1732263678590881, "cf_name": "default", "job": 4363, "event": "table_file_creation", "file_number": 214170, "file_size": 38368878, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702634, "largest_seqno": 3373735744, "table_properties": {"data_size": 32398048, "index_size": 1767488, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25836888, "raw_average_key_size": 28, "raw_value_size": 108380860, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922746, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263675, "oldest_key_time": 0, "file_creation_time": 1732263675, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 214170, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:21:18.625216 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214171.sst) /dev/shm/ToplingTemp/Topling-195935-lPVLcU 2024/11/22-16:21:19.485345 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214171.sst): first pass time = 0.86's, 156.050'MB/sec 2024/11/22-16:21:21.067174 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214171.sst): old prealloc_size = 147639500, real_size = 34965392 2024/11/22-16:21:21.067702 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214171.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922710 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0330 GB avg-val = 117.46 avg-zval = 35.74 2024/11/22-16:21:21.068477 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214171.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:21:21.977804 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214171.sst): second pass time = 1.58's, 68.542'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 42.891'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3449.480'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.294'MB/sec write SST data time = 0.13's, 311.656'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34902.629'MB/sec zip my value time = 1.58's, unzip length = 0.108'GB zip my value throughput = 68.542'MB/sec zip pipeline throughput = 98.021'MB/sec entries = 922710 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.46 avg-zval = 35.74 usrkeys = 922710 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.46 avg-zval = 35.74 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922710 bytes = 3229504 } seq expand size = 6458970 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0330 GB dict = 1.08 MB all = 0.0391 GB } UnZip/Zip{ index = 10.5129 value = 3.2864 dict = 1.00 all = 3.4365 } Zip/UnZip{ index = 0.0951 value = 0.3043 dict = 1.00 all = 0.2910 } ---------------------------- total value len = 0.689112 GB avg = 0.125 KB (by entry num) total key len = 0.154926 GB avg = 0.028 KB total ukey len = 0.110662 GB avg = 0.020 KB total ukey num = 0.005533084 Billion total entry num = 0.005533084 Billion write speed all = 0.031331605 MB/sec (with seq num) write speed all = 0.029688453 MB/sec (without seq num) 2024/11/22-16:21:22.012253 140025179854400 [compaction_job.cc:1948] [default] [JOB 4363] Generated table #214171: 922710 keys, 39056950 bytes, temperature: kWarm 2024/11/22-16:21:22.012310 140025179854400 EVENT_LOG_v1 {"time_micros": 1732263682012280, "cf_name": "default", "job": 4363, "event": "table_file_creation", "file_number": 214171, "file_size": 39056950, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868609, "largest_seqno": 3373735594, "table_properties": {"data_size": 32979136, "index_size": 1755392, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25835880, "raw_average_key_size": 28, "raw_value_size": 108381972, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922710, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263678, "oldest_key_time": 0, "file_creation_time": 1732263678, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 214171, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:21:22.045590 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214172.sst) /dev/shm/ToplingTemp/Topling-195935-3AJlpR 2024/11/22-16:21:22.916634 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214172.sst): first pass time = 0.87's, 154.094'MB/sec 2024/11/22-16:21:24.396989 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214172.sst): old prealloc_size = 147639500, real_size = 36568256 2024/11/22-16:21:24.397506 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214172.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923077 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0346 GB avg-val = 117.40 avg-zval = 37.46 2024/11/22-16:21:24.398247 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214172.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:21:25.239150 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214172.sst): second pass time = 1.48's, 73.270'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 48.004'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3509.179'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 320.213'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33204.563'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.270'MB/sec zip pipeline throughput = 84.476'MB/sec entries = 923077 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.40 avg-zval = 37.46 usrkeys = 923077 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.40 avg-zval = 37.46 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923077 bytes = 3000016 } seq expand size = 6461539 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0346 GB dict = 1.08 MB all = 0.0404 GB } UnZip/Zip{ index = 10.5117 value = 3.1338 dict = 1.00 all = 3.3201 } Zip/UnZip{ index = 0.0951 value = 0.3191 dict = 1.00 all = 0.3012 } ---------------------------- total value len = 0.803945 GB avg = 0.125 KB (by entry num) total key len = 0.180773 GB avg = 0.028 KB total ukey len = 0.129123 GB avg = 0.020 KB total ukey num = 0.006456161 Billion total entry num = 0.006456161 Billion write speed all = 0.036549346 MB/sec (with seq num) write speed all = 0.034632302 MB/sec (without seq num) 2024/11/22-16:21:25.272587 140025179854400 [compaction_job.cc:1948] [default] [JOB 4363] Generated table #214172: 923077 keys, 40426210 bytes, temperature: kWarm 2024/11/22-16:21:25.272645 140025179854400 EVENT_LOG_v1 {"time_micros": 1732263685272614, "cf_name": "default", "job": 4363, "event": "table_file_creation", "file_number": 214172, "file_size": 40426210, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868619, "largest_seqno": 3273700579, "table_properties": {"data_size": 34581040, "index_size": 1756288, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25846156, "raw_average_key_size": 28, "raw_value_size": 108371618, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923077, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263682, "oldest_key_time": 0, "file_creation_time": 1732263682, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 214172, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:21:25.312155 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214173.sst) /dev/shm/ToplingTemp/Topling-195935-O1H8WR 2024/11/22-16:21:26.180789 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214173.sst): first pass time = 0.87's, 154.523'MB/sec 2024/11/22-16:21:27.671139 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214173.sst): old prealloc_size = 147639500, real_size = 36500864 2024/11/22-16:21:27.671677 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214173.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922132 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0345 GB avg-val = 117.55 avg-zval = 37.42 2024/11/22-16:21:27.672386 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214173.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:21:28.545024 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214173.sst): second pass time = 1.49's, 72.763'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 46.084'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3388.797'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.17's, 230.755'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 35309.309'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.763'MB/sec zip pipeline throughput = 85.609'MB/sec entries = 922132 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.55 avg-zval = 37.42 usrkeys = 922132 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.01 avg-zval = 37.42 TagRS{ kinds = RS_Key0_TagN bytes = 144200 } TagArray{ size = 851449 bytes = 2767232 } seq expand size = 5960143 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0403 GB } UnZip/Zip{ index = 10.4446 value = 3.1416 dict = 1.00 all = 3.3328 } Zip/UnZip{ index = 0.0957 value = 0.3183 dict = 1.00 all = 0.3000 } ---------------------------- total value len = 0.918303 GB avg = 0.124 KB (by entry num) total key len = 0.206592 GB avg = 0.028 KB total ukey len = 0.147566 GB avg = 0.020 KB total ukey num = 0.007378293 Billion total entry num = 0.007378293 Billion write speed all = 0.041747146 MB/sec (with seq num) write speed all = 0.039556560 MB/sec (without seq num) 2024/11/22-16:21:28.579286 140025179854400 [compaction_job.cc:1948] [default] [JOB 4363] Generated table #214173: 922132 keys, 40271278 bytes, temperature: kWarm 2024/11/22-16:21:28.579347 140025179854400 EVENT_LOG_v1 {"time_micros": 1732263688579316, "cf_name": "default", "job": 4363, "event": "table_file_creation", "file_number": 214173, "file_size": 40271278, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700489, "table_properties": {"data_size": 34504368, "index_size": 1765760, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25819696, "raw_average_key_size": 28, "raw_value_size": 108398040, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922132, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263685, "oldest_key_time": 0, "file_creation_time": 1732263685, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 214173, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:21:28.615641 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214174.sst) /dev/shm/ToplingTemp/Topling-195935-CTB43Q 2024/11/22-16:21:29.560950 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214174.sst): first pass time = 0.95's, 141.988'MB/sec 2024/11/22-16:21:31.042825 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214174.sst): old prealloc_size = 147639500, real_size = 36716816 2024/11/22-16:21:31.043353 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214174.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926282 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0347 GB avg-val = 116.90 avg-zval = 37.50 2024/11/22-16:21:31.043653 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214174.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:21:31.742053 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214174.sst): second pass time = 1.48's, 73.104'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.70's, 55.093'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3468.072'MB/sec (index lex order gen) rebuild zvType time = 0.57's, 0.404'MB/sec write SST data time = 0.13's, 307.401'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34582.725'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.104'MB/sec zip pipeline throughput = 73.104'MB/sec entries = 926282 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.90 avg-zval = 37.50 usrkeys = 926282 avg-key = 20.00 avg-zkey = 1.89 avg-val = 118.76 avg-zval = 37.50 TagRS{ kinds = RS_Key0_TagN bytes = 144840 } TagArray{ size = 246694 bytes = 801776 } seq expand size = 1726858 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0347 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.5840 value = 3.1174 dict = 1.00 all = 3.4842 } Zip/UnZip{ index = 0.0945 value = 0.3208 dict = 1.00 all = 0.2870 } ---------------------------- total value len = 1.028312 GB avg = 0.124 KB (by entry num) total key len = 0.232528 GB avg = 0.028 KB total ukey len = 0.166092 GB avg = 0.020 KB total ukey num = 0.008304575 Billion total entry num = 0.008304575 Billion write speed all = 0.046786778 MB/sec (with seq num) write speed all = 0.044321474 MB/sec (without seq num) 2024/11/22-16:21:31.780498 140025179854400 [compaction_job.cc:1948] [default] [JOB 4363] Generated table #214174: 926282 keys, 38521649 bytes, temperature: kWarm 2024/11/22-16:21:31.780557 140025179854400 EVENT_LOG_v1 {"time_micros": 1732263691780525, "cf_name": "default", "job": 4363, "event": "table_file_creation", "file_number": 214174, "file_size": 38521649, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700833, "table_properties": {"data_size": 34734720, "index_size": 1750336, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25935896, "raw_average_key_size": 28, "raw_value_size": 108281952, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926282, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263688, "oldest_key_time": 0, "file_creation_time": 1732263688, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 214174, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:21:31.814251 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214175.sst) /dev/shm/ToplingTemp/Topling-195935-40qBbR 2024/11/22-16:21:32.564445 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214175.sst): first pass time = 0.75's, 178.918'MB/sec 2024/11/22-16:21:34.127506 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214175.sst): old prealloc_size = 147639500, real_size = 36458736 2024/11/22-16:21:34.128045 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214175.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927084 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.77 avg-zval = 37.17 2024/11/22-16:21:34.128088 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214175.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:21:34.770483 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214175.sst): second pass time = 1.56's, 69.287'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 58.039'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3413.875'MB/sec (index lex order gen) rebuild zvType time = 0.52's, 0.447'MB/sec write SST data time = 0.12's, 300.693'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33561.891'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.287'MB/sec zip pipeline throughput = 69.287'MB/sec entries = 927084 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.77 avg-zval = 37.17 usrkeys = 927084 avg-key = 20.00 avg-zkey = 1.90 avg-val = 116.77 avg-zval = 37.17 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5121 value = 3.1413 dict = 1.00 all = 3.5967 } Zip/UnZip{ index = 0.0951 value = 0.3183 dict = 1.00 all = 0.2780 } ---------------------------- total value len = 1.136572 GB avg = 0.123 KB (by entry num) total key len = 0.258486 GB avg = 0.028 KB total ukey len = 0.184633 GB avg = 0.020 KB total ukey num = 0.009231659 Billion total entry num = 0.009231659 Billion write speed all = 0.051761462 MB/sec (with seq num) write speed all = 0.049021251 MB/sec (without seq num) 2024/11/22-16:21:34.803598 140025179854400 [compaction_job.cc:1948] [default] [JOB 4363] Generated table #214175: 927084 keys, 37317242 bytes, temperature: kWarm 2024/11/22-16:21:34.803660 140025179854400 EVENT_LOG_v1 {"time_micros": 1732263694803628, "cf_name": "default", "job": 4363, "event": "table_file_creation", "file_number": 214175, "file_size": 37317242, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34462944, "index_size": 1763840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25958352, "raw_average_key_size": 28, "raw_value_size": 108259412, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927084, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263691, "oldest_key_time": 0, "file_creation_time": 1732263691, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 214175, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:21:34.839563 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214176.sst) /dev/shm/ToplingTemp/Topling-195935-GP4x8S 2024/11/22-16:21:35.731190 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214176.sst): first pass time = 0.89's, 151.581'MB/sec 2024/11/22-16:21:37.172014 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214176.sst): old prealloc_size = 147639500, real_size = 36372816 2024/11/22-16:21:37.172562 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214176.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924138 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0344 GB avg-val = 117.24 avg-zval = 37.21 2024/11/22-16:21:37.172935 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214176.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:21:37.910367 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214176.sst): second pass time = 1.44's, 75.227'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.74's, 52.144'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3329.613'MB/sec (index lex order gen) rebuild zvType time = 0.61's, 0.378'MB/sec write SST data time = 0.13's, 304.973'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 34075.346'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.227'MB/sec zip pipeline throughput = 75.227'MB/sec entries = 924138 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.24 avg-zval = 37.21 usrkeys = 924138 avg-key = 20.00 avg-zkey = 1.90 avg-val = 119.75 avg-zval = 37.21 TagRS{ kinds = RS_Key0_TagN bytes = 144480 } TagArray{ size = 332249 bytes = 1121360 } seq expand size = 2325743 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.5119 value = 3.1510 dict = 1.00 all = 3.4863 } Zip/UnZip{ index = 0.0951 value = 0.3174 dict = 1.00 all = 0.2868 } ---------------------------- total value len = 1.247239 GB avg = 0.123 KB (by entry num) total key len = 0.284362 GB avg = 0.028 KB total ukey len = 0.203116 GB avg = 0.020 KB total ukey num = 0.010155797 Billion total entry num = 0.010155797 Billion write speed all = 0.056821081 MB/sec (with seq num) write speed all = 0.053806911 MB/sec (without seq num) 2024/11/22-16:21:37.943817 140025179854400 [compaction_job.cc:1948] [default] [JOB 4363] Generated table #214176: 924138 keys, 38498829 bytes, temperature: kWarm 2024/11/22-16:21:37.943875 140025179854400 EVENT_LOG_v1 {"time_micros": 1732263697943844, "cf_name": "default", "job": 4363, "event": "table_file_creation", "file_number": 214176, "file_size": 38498829, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3341697936, "table_properties": {"data_size": 34383296, "index_size": 1758272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25875864, "raw_average_key_size": 28, "raw_value_size": 108342004, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924138, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263694, "oldest_key_time": 0, "file_creation_time": 1732263694, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 214176, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:21:37.985716 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214177.sst) /dev/shm/ToplingTemp/Topling-195935-TXlb0U 2024/11/22-16:21:38.872378 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214177.sst): first pass time = 0.89's, 151.380'MB/sec 2024/11/22-16:21:40.186178 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214177.sst): old prealloc_size = 147639500, real_size = 34546304 2024/11/22-16:21:40.186725 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214177.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922839 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0326 GB avg-val = 117.44 avg-zval = 35.28 2024/11/22-16:21:40.187650 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214177.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:21:41.082789 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214177.sst): second pass time = 1.31's, 82.530'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 42.962'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3350.260'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.300'MB/sec write SST data time = 0.13's, 304.200'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32992.323'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.530'MB/sec zip pipeline throughput = 82.530'MB/sec entries = 922839 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.44 avg-zval = 35.28 usrkeys = 922839 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.44 avg-zval = 35.28 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922839 bytes = 3114592 } seq expand size = 6459873 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0326 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.5140 value = 3.3286 dict = 1.00 all = 3.4845 } Zip/UnZip{ index = 0.0951 value = 0.3004 dict = 1.00 all = 0.2870 } ---------------------------- total value len = 1.362078 GB avg = 0.123 KB (by entry num) total key len = 0.310202 GB avg = 0.028 KB total ukey len = 0.221573 GB avg = 0.020 KB total ukey num = 0.011078636 Billion total entry num = 0.011078636 Billion write speed all = 0.062032800 MB/sec (with seq num) write speed all = 0.058745126 MB/sec (without seq num) 2024/11/22-16:21:41.118916 140025179854400 [compaction_job.cc:1948] [default] [JOB 4363] Generated table #214177: 922839 keys, 38519054 bytes, temperature: kWarm 2024/11/22-16:21:41.118973 140025179854400 EVENT_LOG_v1 {"time_micros": 1732263701118944, "cf_name": "default", "job": 4363, "event": "table_file_creation", "file_number": 214177, "file_size": 38519054, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701362, "largest_seqno": 3373735950, "table_properties": {"data_size": 32559984, "index_size": 1755456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25839492, "raw_average_key_size": 28, "raw_value_size": 108378362, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922839, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263697, "oldest_key_time": 0, "file_creation_time": 1732263697, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 214177, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:21:41.156008 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214178.sst) /dev/shm/ToplingTemp/Topling-195935-H6BmFS 2024/11/22-16:21:42.031305 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214178.sst): first pass time = 0.88's, 153.345'MB/sec 2024/11/22-16:21:43.353389 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214178.sst): old prealloc_size = 147639500, real_size = 33821648 2024/11/22-16:21:43.353949 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214178.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921644 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0318 GB avg-val = 117.63 avg-zval = 34.54 2024/11/22-16:21:43.355304 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214178.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:21:44.280421 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214178.sst): second pass time = 1.32's, 82.067'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.766'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3263.551'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.288'MB/sec write SST data time = 0.13's, 297.366'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 33151.039'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.067'MB/sec zip pipeline throughput = 82.067'MB/sec entries = 921644 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.63 avg-zval = 34.54 usrkeys = 921644 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.63 avg-zval = 34.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921644 bytes = 3110560 } seq expand size = 6451508 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.5107 value = 3.4052 dict = 1.00 all = 3.5516 } Zip/UnZip{ index = 0.0951 value = 0.2937 dict = 1.00 all = 0.2816 } ---------------------------- total value len = 1.476941 GB avg = 0.123 KB (by entry num) total key len = 0.336008 GB avg = 0.028 KB total ukey len = 0.240006 GB avg = 0.020 KB total ukey num = 0.012000280 Billion total entry num = 0.012000280 Billion write speed all = 0.067242916 MB/sec (with seq num) write speed all = 0.063682159 MB/sec (without seq num) 2024/11/22-16:21:44.313317 140025179854400 [compaction_job.cc:1948] [default] [JOB 4363] Generated table #214178: 921644 keys, 37791208 bytes, temperature: kWarm 2024/11/22-16:21:44.313380 140025179854400 EVENT_LOG_v1 {"time_micros": 1732263704313348, "cf_name": "default", "job": 4363, "event": "table_file_creation", "file_number": 214178, "file_size": 37791208, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701386, "largest_seqno": 3373735975, "table_properties": {"data_size": 31837312, "index_size": 1753728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25806032, "raw_average_key_size": 28, "raw_value_size": 108411708, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921644, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263701, "oldest_key_time": 0, "file_creation_time": 1732263701, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 214178, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:21:44.347876 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214179.sst) /dev/shm/ToplingTemp/Topling-195935-zKLSBT 2024/11/22-16:21:45.174346 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214179.sst): first pass time = 0.83's, 162.408'MB/sec 2024/11/22-16:21:46.494874 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214179.sst): old prealloc_size = 147639500, real_size = 34274352 2024/11/22-16:21:46.495447 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214179.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922336 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0323 GB avg-val = 117.52 avg-zval = 35.01 2024/11/22-16:21:46.496179 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214179.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:21:47.397254 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214179.sst): second pass time = 1.32's, 82.149'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 42.385'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3154.076'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.297'MB/sec write SST data time = 0.12's, 307.196'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 35052.349'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.149'MB/sec zip pipeline throughput = 82.149'MB/sec entries = 922336 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.52 avg-zval = 35.01 usrkeys = 922336 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.52 avg-zval = 35.01 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922336 bytes = 3112896 } seq expand size = 6456352 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0323 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.5128 value = 3.3570 dict = 1.00 all = 3.5093 } Zip/UnZip{ index = 0.0951 value = 0.2979 dict = 1.00 all = 0.2850 } ---------------------------- total value len = 1.591790 GB avg = 0.123 KB (by entry num) total key len = 0.361833 GB avg = 0.028 KB total ukey len = 0.258452 GB avg = 0.020 KB total ukey num = 0.012922616 Billion total entry num = 0.012922616 Billion write speed all = 0.072452197 MB/sec (with seq num) write speed all = 0.068618204 MB/sec (without seq num) 2024/11/22-16:21:47.428596 140025179854400 [compaction_job.cc:1948] [default] [JOB 4363] Generated table #214179: 922336 keys, 38245974 bytes, temperature: kWarm 2024/11/22-16:21:47.428653 140025179854400 EVENT_LOG_v1 {"time_micros": 1732263707428623, "cf_name": "default", "job": 4363, "event": "table_file_creation", "file_number": 214179, "file_size": 38245974, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701396, "largest_seqno": 3373735993, "table_properties": {"data_size": 32288928, "index_size": 1754688, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25825408, "raw_average_key_size": 28, "raw_value_size": 108392444, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922336, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263704, "oldest_key_time": 0, "file_creation_time": 1732263704, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 214179, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:21:47.464368 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214180.sst) /dev/shm/ToplingTemp/Topling-195935-e1TnYT 2024/11/22-16:21:47.535453 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214180.sst): first pass time = 0.07's, 239.795'MB/sec 2024/11/22-16:21:47.695847 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214180.sst): old prealloc_size = 147639500, real_size = 5102192 2024/11/22-16:21:47.695958 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214180.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 116709 prefix = 4 raw-key = 0.0019 GB zip-key = 0.0002 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0138 GB zip-val = 0.0048 GB avg-val = 118.00 avg-zval = 41.54 2024/11/22-16:21:47.696071 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214180.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0002 GB, waited 0.000 sec, Key+Value = 0.017 GB 2024/11/22-16:21:47.802906 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00/214180.sst): second pass time = 0.16's, 86.119'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.11's, 52.156'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2435.837'MB/sec (index lex order gen) rebuild zvType time = 0.09's, 0.321'MB/sec write SST data time = 0.02's, 347.695'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.144'MB, throughput = 5861.833'MB/sec zip my value time = 0.16's, unzip length = 0.014'GB zip my value throughput = 86.119'MB/sec zip pipeline throughput = 86.119'MB/sec entries = 116709 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 41.54 usrkeys = 116709 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 41.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 116709 bytes = 364736 } seq expand size = 816963 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0023 GB value = 0.0138 GB dict = 0.14 MB all = 0.0170 GB } __ZipSize{ index = 0.0002 GB value = 0.0048 GB dict = 0.14 MB all = 0.0056 GB } UnZip/Zip{ index = 10.3789 value = 2.8407 dict = 1.00 all = 3.0519 } Zip/UnZip{ index = 0.0963 value = 0.3520 dict = 1.00 all = 0.3277 } ---------------------------- total value len = 1.606378 GB avg = 0.123 KB (by entry num) total key len = 0.365101 GB avg = 0.028 KB total ukey len = 0.260786 GB avg = 0.020 KB total ukey num = 0.013039325 Billion total entry num = 0.013039325 Billion write speed all = 0.073113324 MB/sec (with seq num) write speed all = 0.069244763 MB/sec (without seq num) 2024/11/22-16:21:47.815109 140025179854400 [compaction_job.cc:1948] [default] [JOB 4363] Generated table #214180: 116709 keys, 5583277 bytes, temperature: kWarm 2024/11/22-16:21:47.815161 140025179854400 EVENT_LOG_v1 {"time_micros": 1732263707815132, "cf_name": "default", "job": 4363, "event": "table_file_creation", "file_number": 214180, "file_size": 5583277, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3339289870, "largest_seqno": 3362982790, "table_properties": {"data_size": 4847904, "index_size": 224896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3267852, "raw_average_key_size": 28, "raw_value_size": 13771662, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 116709, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263707, "oldest_key_time": 0, "file_creation_time": 1732263707, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 214180, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:21:47.827475 140025179854400 [compaction_job.cc:703] job-04363: subcompact[0], size: 0.545553 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04363/att-00 [214166,214167,214168,214169,214170,214171,214172,214173,214174,214175,214176,214177,214178,214179,214180] 2024/11/22-16:21:47.919816 140025179854400 (Original Log Time 2024/11/22-16:21:01.399585) EVENT_LOG_v1 {"time_micros": 1732263661399538, "job": 4363, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [214117], "files_L3": [214065, 214066, 214067, 214068, 214069, 214070, 214071, 214072, 214073, 214074, 214075, 214076, 214082, 214156], "score": 0.878362, "input_data_size": 568083050, "oldest_snapshot_seqno": 3374158245} 2024/11/22-16:21:47.919819 140025179854400 (Original Log Time 2024/11/22-16:21:47.919065) [compaction_job.cc:2022] [default] [JOB 4363] Compacted 1@2 + 14@3 files to L3 => 545553121 bytes 2024/11/22-16:21:47.919820 140025179854400 (Original Log Time 2024/11/22-16:21:47.919747) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 12.2 rd, 11.8 wr, level 3, files in(1, 14) out(15 +0 blob) MB in(27.1, 514.6 +0.0 blob) out(520.3 +0.0 blob), read-write-amplify(39.1) write-amplify(19.2) OK, records in: 13155459, records dropped: 116134 output_compression: Snappy 2024/11/22-16:21:47.919826 140025179854400 (Original Log Time 2024/11/22-16:21:47.919798) EVENT_LOG_v1 {"time_micros": 1732263707919771, "cf": "default", "job": 4363, "event": "compaction_finished", "compaction_time_micros": 46427894, "compaction_time_cpu_micros": 19821268, "output_level": 3, "num_output_files": 15, "total_output_size": 545553121, "num_input_records": 13155459, "num_output_records": 13039325, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 341199057, "file_range_sync_nanos": 0, "file_fsync_nanos": 13026, "file_prepare_write_nanos": 32485, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/22-16:21:48.053165 140025179854400 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732263661,"executesMs":46644,"compactionJobId":4363,"attempt":0,"compactionInputRawBytes":1912579798,"compactionInputZipBytes":568083050,"compactionOutputRawBytes":0,"compactionOutputZipBytes":545553121,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-47a49c67a3ff553c7359ceaf6754654c-dfe5d9d1f1447f07-00"}}, response = 2024/11/22-16:21:48.053227 140025179854400 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-04363/att-00: olev 3, work 46.645 s, result 4.889 ms, install 0.784 ms, input{raw 1.781 GiB zip 541.766 MiB}