2024/11/23-11:38:37.538547 139957416588864 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00: fork to child time = 0.008251 sec 2024/11/23-11:38:37.539101 139957416588864 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00 2024/11/23-11:38:37.545728 139957416588864 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00 2024/11/23-11:38:37.546640 139957416588864 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-11:38:37.546816 139957416588864 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-11:38:37.600719 139957416588864 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-25348/att-00/MANIFEST-206347 2024/11/23-11:38:37.634297 139957416588864 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-25348/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 258074, last_sequence is 3956107470, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 257854 2024/11/23-11:38:37.634303 139957416588864 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-11:38:37.634384 139957416588864 [version_set.cc:5721] Creating manifest 258074 2024/11/23-11:38:37.675259 139957416588864 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-11:38:37.695327 139957416588864 [compaction_job.cc:2437] [default] [JOB 25348] Compacting 1@2 + 15@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-11:38:37.695339 139957416588864 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 18994 Base level 2, inputs: [257999(27MB)], [258056(33MB) 258057(7628KB) 257956(34MB) 257957(35MB) 257958(35MB) 257959(35MB) 257960(33MB) 257961(35MB) 257962(36MB) 257964(36MB) 257965(34MB) 257966(35MB) 257967(35MB) 257968(35MB) 258058(35MB)] 2024/11/23-11:38:37.702649 139957416588864 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-11:38:37.702676 139957416588864 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-11:38:37.702691 139957416588864 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-11:38:37.702696 139957416588864 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-11:38:37.702703 139957416588864 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-11:38:38.379584 139957416588864 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258075.sst) /dev/shm/ToplingTemp/Topling-38898-mzOZ4r 2024/11/23-11:38:38.929827 139957416588864 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258075.sst): first pass time = 0.55's, 243.944'MB/sec 2024/11/23-11:38:40.346102 139957416588864 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258075.sst): old prealloc_size = 147639500, real_size = 30509104 2024/11/23-11:38:40.346681 139957416588864 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258075.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931034 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0285 GB avg-val = 116.16 avg-zval = 30.60 2024/11/23-11:38:40.347522 139957416588864 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258075.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:38:41.179338 139957416588864 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258075.sst): second pass time = 1.42's, 76.427'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 41.689'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3184.582'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 271.549'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33238.034'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 76.427'MB/sec zip pipeline throughput = 76.427'MB/sec entries = 931034 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.16 avg-zval = 30.60 usrkeys = 931034 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.16 avg-zval = 30.60 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931034 bytes = 3375008 } seq expand size = 6517238 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0285 GB dict = 1.08 MB all = 0.0347 GB } UnZip/Zip{ index = 10.4455 value = 3.7956 dict = 1.00 all = 3.8640 } Zip/UnZip{ index = 0.0957 value = 0.2635 dict = 1.00 all = 0.2588 } ---------------------------- total value len = 8.963286 GB avg = 0.168 KB (by entry num) total key len = 1.466890 GB avg = 0.028 KB total ukey len = 1.016499 GB avg = 0.015 KB total ukey num = 0.066481469 Billion total entry num = 0.053289198 Billion write speed all = 1.063330973 MB/sec (with seq num) write speed all = 1.019869344 MB/sec (without seq num) 2024/11/23-11:38:41.221948 139957416588864 [compaction_job.cc:1948] [default] [JOB 25348] Generated table #258075: 931034 keys, 34735832 bytes, temperature: kWarm 2024/11/23-11:38:41.222067 139957416588864 EVENT_LOG_v1 {"time_micros": 1732333121221998, "cf_name": "default", "job": 25348, "event": "table_file_creation", "file_number": 258075, "file_size": 34735832, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 3955702312, "table_properties": {"data_size": 28493536, "index_size": 1782656, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26068952, "raw_average_key_size": 28, "raw_value_size": 108148896, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931034, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732333118, "oldest_key_time": 0, "file_creation_time": 1732333118, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 258075, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:38:41.257460 139957416588864 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258076.sst) /dev/shm/ToplingTemp/Topling-38898-ljXJ5t 2024/11/23-11:38:42.226620 139957416588864 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258076.sst): first pass time = 0.97's, 138.493'MB/sec 2024/11/23-11:38:43.527970 139957416588864 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258076.sst): old prealloc_size = 147639500, real_size = 31757744 2024/11/23-11:38:43.528526 139957416588864 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258076.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930207 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0297 GB avg-val = 116.29 avg-zval = 31.98 2024/11/23-11:38:43.529313 139957416588864 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258076.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:38:44.369066 139957416588864 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258076.sst): second pass time = 1.30's, 83.178'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.781'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3481.539'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 279.337'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35542.754'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.178'MB/sec zip pipeline throughput = 83.178'MB/sec entries = 930207 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.29 avg-zval = 31.98 usrkeys = 930207 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.29 avg-zval = 31.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930207 bytes = 3372016 } seq expand size = 6511449 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.4441 value = 3.6368 dict = 1.00 all = 3.7301 } Zip/UnZip{ index = 0.0957 value = 0.2750 dict = 1.00 all = 0.2681 } ---------------------------- total value len = 9.077969 GB avg = 0.167 KB (by entry num) total key len = 1.492936 GB avg = 0.028 KB total ukey len = 1.035103 GB avg = 0.015 KB total ukey num = 0.067411676 Billion total entry num = 0.054219405 Billion write speed all = 1.077327650 MB/sec (with seq num) write speed all = 1.033121737 MB/sec (without seq num) 2024/11/23-11:38:44.420194 139957416588864 [compaction_job.cc:1948] [default] [JOB 25348] Generated table #258076: 930207 keys, 35982134 bytes, temperature: kWarm 2024/11/23-11:38:44.420264 139957416588864 EVENT_LOG_v1 {"time_micros": 1732333124420225, "cf_name": "default", "job": 25348, "event": "table_file_creation", "file_number": 258076, "file_size": 35982134, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3585662922, "largest_seqno": 3955702390, "table_properties": {"data_size": 29743712, "index_size": 1781312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26045796, "raw_average_key_size": 28, "raw_value_size": 108172050, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930207, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732333121, "oldest_key_time": 0, "file_creation_time": 1732333121, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 258076, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:38:44.457916 139957416588864 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258077.sst) /dev/shm/ToplingTemp/Topling-38898-lsWVlv 2024/11/23-11:38:45.309546 139957416588864 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258077.sst): first pass time = 0.85's, 157.608'MB/sec 2024/11/23-11:38:46.641673 139957416588864 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258077.sst): old prealloc_size = 147639500, real_size = 32948416 2024/11/23-11:38:46.642278 139957416588864 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258077.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930459 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.25 avg-zval = 33.25 2024/11/23-11:38:46.643121 139957416588864 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258077.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:38:47.500237 139957416588864 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258077.sst): second pass time = 1.33's, 81.248'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.433'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3055.218'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 293.373'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 30839.291'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.248'MB/sec zip pipeline throughput = 81.248'MB/sec entries = 930459 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.25 avg-zval = 33.25 usrkeys = 930459 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.25 avg-zval = 33.25 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930459 bytes = 3489232 } seq expand size = 6513213 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4450 value = 3.4966 dict = 1.00 all = 3.5994 } Zip/UnZip{ index = 0.0957 value = 0.2860 dict = 1.00 all = 0.2778 } ---------------------------- total value len = 9.192648 GB avg = 0.167 KB (by entry num) total key len = 1.518989 GB avg = 0.028 KB total ukey len = 1.053712 GB avg = 0.015 KB total ukey num = 0.068342135 Billion total entry num = 0.055149864 Billion write speed all = 1.091321912 MB/sec (with seq num) write speed all = 1.046371726 MB/sec (without seq num) 2024/11/23-11:38:47.552049 139957416588864 [compaction_job.cc:1948] [default] [JOB 25348] Generated table #258077: 930459 keys, 37289346 bytes, temperature: kWarm 2024/11/23-11:38:47.552117 139957416588864 EVENT_LOG_v1 {"time_micros": 1732333127552082, "cf_name": "default", "job": 25348, "event": "table_file_creation", "file_number": 258077, "file_size": 37289346, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3955702210, "table_properties": {"data_size": 30934000, "index_size": 1781632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26052852, "raw_average_key_size": 28, "raw_value_size": 108164914, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930459, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732333124, "oldest_key_time": 0, "file_creation_time": 1732333124, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 258077, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:38:47.588802 139957416588864 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258078.sst) /dev/shm/ToplingTemp/Topling-38898-LwQ8Mt 2024/11/23-11:38:48.489688 139957416588864 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258078.sst): first pass time = 0.90's, 148.992'MB/sec 2024/11/23-11:38:49.814184 139957416588864 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258078.sst): old prealloc_size = 147639500, real_size = 32844416 2024/11/23-11:38:49.814770 139957416588864 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258078.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930388 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0308 GB avg-val = 116.26 avg-zval = 33.15 2024/11/23-11:38:49.815580 139957416588864 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258078.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:38:50.667258 139957416588864 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258078.sst): second pass time = 1.32's, 81.715'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.455'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3146.006'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.13's, 294.213'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 31417.380'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.715'MB/sec zip pipeline throughput = 81.715'MB/sec entries = 930388 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.26 avg-zval = 33.15 usrkeys = 930388 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.26 avg-zval = 33.15 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930388 bytes = 3372672 } seq expand size = 6512716 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5130 value = 3.5072 dict = 1.00 all = 3.6207 } Zip/UnZip{ index = 0.0951 value = 0.2851 dict = 1.00 all = 0.2762 } ---------------------------- total value len = 9.307327 GB avg = 0.166 KB (by entry num) total key len = 1.545040 GB avg = 0.028 KB total ukey len = 1.072320 GB avg = 0.015 KB total ukey num = 0.069272523 Billion total entry num = 0.056080252 Billion write speed all = 1.105303170 MB/sec (with seq num) write speed all = 1.059609410 MB/sec (without seq num) 2024/11/23-11:38:50.717746 139957416588864 [compaction_job.cc:1948] [default] [JOB 25348] Generated table #258078: 930388 keys, 37069162 bytes, temperature: kWarm 2024/11/23-11:38:50.717815 139957416588864 EVENT_LOG_v1 {"time_micros": 1732333130717780, "cf_name": "default", "job": 25348, "event": "table_file_creation", "file_number": 258078, "file_size": 37069162, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3563652023, "largest_seqno": 3955702227, "table_properties": {"data_size": 30841648, "index_size": 1769984, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26050864, "raw_average_key_size": 28, "raw_value_size": 108166976, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930388, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732333127, "oldest_key_time": 0, "file_creation_time": 1732333127, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 258078, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:38:50.755754 139957416588864 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258079.sst) /dev/shm/ToplingTemp/Topling-38898-gIps8s 2024/11/23-11:38:51.771240 139957416588864 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258079.sst): first pass time = 1.02's, 132.177'MB/sec 2024/11/23-11:38:53.074688 139957416588864 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258079.sst): old prealloc_size = 147639500, real_size = 31265984 2024/11/23-11:38:53.075294 139957416588864 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258079.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930635 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0293 GB avg-val = 116.22 avg-zval = 31.44 2024/11/23-11:38:53.076072 139957416588864 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258079.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:38:53.903612 139957416588864 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258079.sst): second pass time = 1.30's, 83.038'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.677'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3031.675'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.13's, 281.527'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 25629.445'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.038'MB/sec zip pipeline throughput = 83.038'MB/sec entries = 930635 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.22 avg-zval = 31.44 usrkeys = 930635 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.22 avg-zval = 31.44 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930635 bytes = 3257232 } seq expand size = 6514445 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.08 MB all = 0.0354 GB } UnZip/Zip{ index = 10.5135 value = 3.6962 dict = 1.00 all = 3.7942 } Zip/UnZip{ index = 0.0951 value = 0.2706 dict = 1.00 all = 0.2636 } ---------------------------- total value len = 9.422002 GB avg = 0.165 KB (by entry num) total key len = 1.571097 GB avg = 0.028 KB total ukey len = 1.090932 GB avg = 0.016 KB total ukey num = 0.070203158 Billion total entry num = 0.057010887 Billion write speed all = 1.119267689 MB/sec (with seq num) write speed all = 1.072830961 MB/sec (without seq num) 2024/11/23-11:38:53.950918 139957416588864 [compaction_job.cc:1948] [default] [JOB 25348] Generated table #258079: 930635 keys, 35374682 bytes, temperature: kWarm 2024/11/23-11:38:53.951008 139957416588864 EVENT_LOG_v1 {"time_micros": 1732333133950954, "cf_name": "default", "job": 25348, "event": "table_file_creation", "file_number": 258079, "file_size": 35374682, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3759280178, "largest_seqno": 3955702238, "table_properties": {"data_size": 29262768, "index_size": 1770368, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26057780, "raw_average_key_size": 28, "raw_value_size": 108160090, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930635, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732333130, "oldest_key_time": 0, "file_creation_time": 1732333130, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 258079, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:38:53.991697 139957416588864 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258080.sst) /dev/shm/ToplingTemp/Topling-38898-rplj3u 2024/11/23-11:38:54.926417 139957416588864 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258080.sst): first pass time = 0.93's, 143.597'MB/sec 2024/11/23-11:38:56.190133 139957416588864 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258080.sst): old prealloc_size = 147639500, real_size = 30916576 2024/11/23-11:38:56.190698 139957416588864 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258080.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925107 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0289 GB avg-val = 117.08 avg-zval = 31.24 2024/11/23-11:38:56.191477 139957416588864 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258080.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:38:57.005439 139957416588864 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258080.sst): second pass time = 1.26's, 85.767'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.940'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3293.747'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.337'MB/sec write SST data time = 0.13's, 273.918'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 36039.737'MB/sec zip my value time = 1.26's, unzip length = 0.108'GB zip my value throughput = 85.767'MB/sec zip pipeline throughput = 85.767'MB/sec entries = 925107 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.08 avg-zval = 31.24 usrkeys = 925107 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.08 avg-zval = 31.24 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925107 bytes = 3237888 } seq expand size = 6475749 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.3775 value = 3.7476 dict = 1.00 all = 3.8339 } Zip/UnZip{ index = 0.0964 value = 0.2668 dict = 1.00 all = 0.2608 } ---------------------------- total value len = 9.536792 GB avg = 0.165 KB (by entry num) total key len = 1.597000 GB avg = 0.028 KB total ukey len = 1.109435 GB avg = 0.016 KB total ukey num = 0.071128265 Billion total entry num = 0.057935994 Billion write speed all = 1.133234583 MB/sec (with seq num) write speed all = 1.086059233 MB/sec (without seq num) 2024/11/23-11:38:57.050953 139957416588864 [compaction_job.cc:1948] [default] [JOB 25348] Generated table #258080: 925107 keys, 35008332 bytes, temperature: kWarm 2024/11/23-11:38:57.051043 139957416588864 EVENT_LOG_v1 {"time_micros": 1732333137051006, "cf_name": "default", "job": 25348, "event": "table_file_creation", "file_number": 258080, "file_size": 35008332, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3775427739, "largest_seqno": 3955702255, "table_properties": {"data_size": 28902224, "index_size": 1782912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25902996, "raw_average_key_size": 28, "raw_value_size": 108314838, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925107, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732333133, "oldest_key_time": 0, "file_creation_time": 1732333133, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 258080, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:38:57.099491 139957416588864 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258081.sst) /dev/shm/ToplingTemp/Topling-38898-7l9a4t 2024/11/23-11:38:58.109661 139957416588864 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258081.sst): first pass time = 1.01's, 132.874'MB/sec 2024/11/23-11:38:59.486188 139957416588864 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258081.sst): old prealloc_size = 147639500, real_size = 35543632 2024/11/23-11:38:59.486726 139957416588864 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258081.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931606 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0335 GB avg-val = 116.07 avg-zval = 36.00 2024/11/23-11:38:59.487263 139957416588864 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258081.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:39:00.264246 139957416588864 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258081.sst): second pass time = 1.38's, 78.601'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.78's, 49.271'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3431.096'MB/sec (index lex order gen) rebuild zvType time = 0.65's, 0.358'MB/sec write SST data time = 0.13's, 299.938'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 29965.360'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.601'MB/sec zip pipeline throughput = 78.601'MB/sec entries = 931606 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.07 avg-zval = 36.00 usrkeys = 931606 avg-key = 20.00 avg-zkey = 1.90 avg-val = 119.92 avg-zval = 36.00 TagRS{ kinds = RS_Key0_TagN bytes = 145680 } TagArray{ size = 511560 bytes = 1790480 } seq expand size = 3580920 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0335 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.5153 value = 3.2241 dict = 1.00 all = 3.5012 } Zip/UnZip{ index = 0.0951 value = 0.3102 dict = 1.00 all = 0.2856 } ---------------------------- total value len = 9.648506 GB avg = 0.164 KB (by entry num) total key len = 1.623085 GB avg = 0.028 KB total ukey len = 1.128067 GB avg = 0.016 KB total ukey num = 0.072059871 Billion total entry num = 0.058867600 Billion write speed all = 1.146879792 MB/sec (with seq num) write speed all = 1.098961760 MB/sec (without seq num) 2024/11/23-11:39:00.313428 139957416588864 [compaction_job.cc:1948] [default] [JOB 25348] Generated table #258081: 931606 keys, 38334381 bytes, temperature: kWarm 2024/11/23-11:39:00.313497 139957416588864 EVENT_LOG_v1 {"time_micros": 1732333140313462, "cf_name": "default", "job": 25348, "event": "table_file_creation", "file_number": 258081, "file_size": 38334381, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3955702268, "table_properties": {"data_size": 33538624, "index_size": 1771904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26084968, "raw_average_key_size": 28, "raw_value_size": 108132900, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931606, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732333137, "oldest_key_time": 0, "file_creation_time": 1732333137, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 258081, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:39:00.350841 139957416588864 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258082.sst) /dev/shm/ToplingTemp/Topling-38898-FUItEu 2024/11/23-11:39:01.424258 139957416588864 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258082.sst): first pass time = 1.07's, 125.044'MB/sec 2024/11/23-11:39:02.807800 139957416588864 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258082.sst): old prealloc_size = 147639500, real_size = 35452848 2024/11/23-11:39:02.808431 139957416588864 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258082.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1080669 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1040 GB zip-val = 0.0331 GB avg-val = 96.20 avg-zval = 30.64 2024/11/23-11:39:02.808663 139957416588864 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258082.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:39:03.499284 139957416588864 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258082.sst): second pass time = 1.38's, 75.189'MB/sec, value only(77.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.69's, 53.176'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3385.033'MB/sec (index lex order gen) rebuild zvType time = 0.55's, 0.488'MB/sec write SST data time = 0.14's, 268.205'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.028'MB, throughput = 32726.778'MB/sec zip my value time = 1.38's, unzip length = 0.104'GB zip my value throughput = 75.189'MB/sec zip pipeline throughput = 75.189'MB/sec entries = 1080669 avg-key = 28.00 avg-zkey = 1.91 avg-val = 96.20 avg-zval = 30.64 usrkeys = 1080669 avg-key = 20.00 avg-zkey = 1.91 avg-val = 96.93 avg-zval = 30.64 TagRS{ kinds = RS_Key0_TagN bytes = 168960 } TagArray{ size = 112312 bytes = 379072 } seq expand size = 786184 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0216 GB value = 0.1040 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0331 GB dict = 1.03 MB all = 0.0368 GB } UnZip/Zip{ index = 10.4473 value = 3.1395 dict = 1.00 all = 3.6502 } Zip/UnZip{ index = 0.0957 value = 0.3185 dict = 1.00 all = 0.2740 } ---------------------------- total value len = 9.753251 GB avg = 0.163 KB (by entry num) total key len = 1.653344 GB avg = 0.028 KB total ukey len = 1.149680 GB avg = 0.016 KB total ukey num = 0.073140540 Billion total entry num = 0.059948269 Billion write speed all = 1.160234485 MB/sec (with seq num) write speed all = 1.111452849 MB/sec (without seq num) 2024/11/23-11:39:03.545697 139957416588864 [compaction_job.cc:1948] [default] [JOB 25348] Generated table #258082: 1080669 keys, 36769534 bytes, temperature: kWarm 2024/11/23-11:39:03.545768 139957416588864 EVENT_LOG_v1 {"time_micros": 1732333143545730, "cf_name": "default", "job": 25348, "event": "table_file_creation", "file_number": 258082, "file_size": 36769534, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3402894352, "table_properties": {"data_size": 33113696, "index_size": 2068800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30258732, "raw_average_key_size": 28, "raw_value_size": 103959071, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1080669, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732333140, "oldest_key_time": 0, "file_creation_time": 1732333140, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 258082, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:39:03.585864 139957416588864 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258083.sst) /dev/shm/ToplingTemp/Topling-38898-50wWov 2024/11/23-11:39:04.571196 139957416588864 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258083.sst): first pass time = 0.99's, 136.221'MB/sec 2024/11/23-11:39:05.898179 139957416588864 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258083.sst): old prealloc_size = 147639500, real_size = 32953136 2024/11/23-11:39:05.898716 139957416588864 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258083.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931499 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0309 GB avg-val = 116.09 avg-zval = 33.22 2024/11/23-11:39:05.899526 139957416588864 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258083.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:39:06.753695 139957416588864 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258083.sst): second pass time = 1.33's, 81.542'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.598'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3457.134'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 298.085'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35363.943'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.542'MB/sec zip pipeline throughput = 81.542'MB/sec entries = 931499 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.09 avg-zval = 33.22 usrkeys = 931499 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.09 avg-zval = 33.22 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931499 bytes = 3493136 } seq expand size = 6520493 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5145 value = 3.4941 dict = 1.00 all = 3.5986 } Zip/UnZip{ index = 0.0951 value = 0.2862 dict = 1.00 all = 0.2779 } ---------------------------- total value len = 9.867908 GB avg = 0.162 KB (by entry num) total key len = 1.679426 GB avg = 0.028 KB total ukey len = 1.168310 GB avg = 0.016 KB total ukey num = 0.074072039 Billion total entry num = 0.060879768 Billion write speed all = 1.174161165 MB/sec (with seq num) write speed all = 1.124637934 MB/sec (without seq num) 2024/11/23-11:39:06.800971 139957416588864 [compaction_job.cc:1948] [default] [JOB 25348] Generated table #258083: 931499 keys, 37297562 bytes, temperature: kWarm 2024/11/23-11:39:06.801051 139957416588864 EVENT_LOG_v1 {"time_micros": 1732333146801016, "cf_name": "default", "job": 25348, "event": "table_file_creation", "file_number": 258083, "file_size": 37297562, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3402894353, "largest_seqno": 3955702760, "table_properties": {"data_size": 30948256, "index_size": 1771840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26081972, "raw_average_key_size": 28, "raw_value_size": 108135894, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931499, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732333143, "oldest_key_time": 0, "file_creation_time": 1732333143, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 258083, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:39:06.835461 139957416588864 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258084.sst) /dev/shm/ToplingTemp/Topling-38898-LsnVzu 2024/11/23-11:39:07.784214 139957416588864 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258084.sst): first pass time = 0.95's, 141.472'MB/sec 2024/11/23-11:39:09.099209 139957416588864 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258084.sst): old prealloc_size = 147639500, real_size = 33667472 2024/11/23-11:39:09.099746 139957416588864 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258084.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922638 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0317 GB avg-val = 117.47 avg-zval = 34.34 2024/11/23-11:39:09.101338 139957416588864 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258084.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:39:09.949198 139957416588864 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258084.sst): second pass time = 1.31's, 82.493'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.691'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3415.183'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.319'MB/sec write SST data time = 0.13's, 301.899'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 35659.588'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.493'MB/sec zip pipeline throughput = 82.493'MB/sec entries = 922638 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.47 avg-zval = 34.34 usrkeys = 922638 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.47 avg-zval = 34.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922638 bytes = 3459904 } seq expand size = 6458466 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.5113 value = 3.4211 dict = 1.00 all = 3.5334 } Zip/UnZip{ index = 0.0951 value = 0.2923 dict = 1.00 all = 0.2830 } ---------------------------- total value len = 9.982750 GB avg = 0.162 KB (by entry num) total key len = 1.705260 GB avg = 0.028 KB total ukey len = 1.186763 GB avg = 0.016 KB total ukey num = 0.074994677 Billion total entry num = 0.061802406 Billion write speed all = 1.188079431 MB/sec (with seq num) write speed all = 1.137822002 MB/sec (without seq num) 2024/11/23-11:39:09.996641 139957416588864 [compaction_job.cc:1948] [default] [JOB 25348] Generated table #258084: 922638 keys, 37985704 bytes, temperature: kWarm 2024/11/23-11:39:09.996708 139957416588864 EVENT_LOG_v1 {"time_micros": 1732333149996672, "cf_name": "default", "job": 25348, "event": "table_file_creation", "file_number": 258084, "file_size": 37985704, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3955702775, "table_properties": {"data_size": 31681088, "index_size": 1755520, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25833864, "raw_average_key_size": 28, "raw_value_size": 108383912, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922638, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732333146, "oldest_key_time": 0, "file_creation_time": 1732333146, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 258084, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:39:10.038427 139957416588864 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258085.sst) /dev/shm/ToplingTemp/Topling-38898-ymaKTr 2024/11/23-11:39:10.962812 139957416588864 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258085.sst): first pass time = 0.92's, 145.203'MB/sec 2024/11/23-11:39:12.264827 139957416588864 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258085.sst): old prealloc_size = 147639500, real_size = 32447440 2024/11/23-11:39:12.265354 139957416588864 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258085.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930817 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0304 GB avg-val = 116.19 avg-zval = 32.69 2024/11/23-11:39:12.266168 139957416588864 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258085.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:39:13.111169 139957416588864 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258085.sst): second pass time = 1.30's, 83.121'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.333'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3537.113'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 290.953'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 24366.680'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.121'MB/sec zip pipeline throughput = 83.121'MB/sec entries = 930817 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.19 avg-zval = 32.69 usrkeys = 930817 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.19 avg-zval = 32.69 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930817 bytes = 3374224 } seq expand size = 6515719 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.4438 value = 3.5540 dict = 1.00 all = 3.6598 } Zip/UnZip{ index = 0.0958 value = 0.2814 dict = 1.00 all = 0.2732 } ---------------------------- total value len = 10.097421 GB avg = 0.161 KB (by entry num) total key len = 1.731323 GB avg = 0.028 KB total ukey len = 1.205379 GB avg = 0.016 KB total ukey num = 0.075925494 Billion total entry num = 0.062733223 Billion write speed all = 1.201998575 MB/sec (with seq num) write speed all = 1.151000601 MB/sec (without seq num) 2024/11/23-11:39:13.157118 139957416588864 [compaction_job.cc:1948] [default] [JOB 25348] Generated table #258085: 930817 keys, 36673286 bytes, temperature: kWarm 2024/11/23-11:39:13.157186 139957416588864 EVENT_LOG_v1 {"time_micros": 1732333153157150, "cf_name": "default", "job": 25348, "event": "table_file_creation", "file_number": 258085, "file_size": 36673286, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3680980241, "largest_seqno": 3955702806, "table_properties": {"data_size": 30432000, "index_size": 1782528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26062876, "raw_average_key_size": 28, "raw_value_size": 108154930, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930817, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732333150, "oldest_key_time": 0, "file_creation_time": 1732333150, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 258085, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:39:13.192889 139957416588864 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258086.sst) /dev/shm/ToplingTemp/Topling-38898-xJFG9u 2024/11/23-11:39:14.061725 139957416588864 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258086.sst): first pass time = 0.87's, 154.486'MB/sec 2024/11/23-11:39:15.346609 139957416588864 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258086.sst): old prealloc_size = 147639500, real_size = 32526352 2024/11/23-11:39:15.347158 139957416588864 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258086.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929024 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0305 GB avg-val = 116.47 avg-zval = 32.85 2024/11/23-11:39:15.349128 139957416588864 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258086.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:39:16.189755 139957416588864 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258086.sst): second pass time = 1.28's, 84.274'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.722'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3367.630'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.13's, 292.596'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 36084.735'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.274'MB/sec zip pipeline throughput = 84.274'MB/sec entries = 929024 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.47 avg-zval = 32.85 usrkeys = 929024 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.47 avg-zval = 32.85 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929024 bytes = 3483856 } seq expand size = 6503168 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4428 value = 3.5460 dict = 1.00 all = 3.6410 } Zip/UnZip{ index = 0.0958 value = 0.2820 dict = 1.00 all = 0.2747 } ---------------------------- total value len = 10.212129 GB avg = 0.160 KB (by entry num) total key len = 1.757335 GB avg = 0.028 KB total ukey len = 1.223960 GB avg = 0.016 KB total ukey num = 0.076854518 Billion total entry num = 0.063662247 Billion write speed all = 1.215917799 MB/sec (with seq num) write speed all = 1.164180775 MB/sec (without seq num) 2024/11/23-11:39:16.237068 139957416588864 [compaction_job.cc:1948] [default] [JOB 25348] Generated table #258086: 929024 keys, 36862938 bytes, temperature: kWarm 2024/11/23-11:39:16.237134 139957416588864 EVENT_LOG_v1 {"time_micros": 1732333156237100, "cf_name": "default", "job": 25348, "event": "table_file_creation", "file_number": 258086, "file_size": 36862938, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369194706, "largest_seqno": 3955702824, "table_properties": {"data_size": 30514624, "index_size": 1779264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26012672, "raw_average_key_size": 28, "raw_value_size": 108205108, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929024, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732333153, "oldest_key_time": 0, "file_creation_time": 1732333153, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 258086, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:39:16.273582 139957416588864 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258087.sst) /dev/shm/ToplingTemp/Topling-38898-kChPAv 2024/11/23-11:39:17.204013 139957416588864 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258087.sst): first pass time = 0.93's, 144.259'MB/sec 2024/11/23-11:39:18.514289 139957416588864 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258087.sst): old prealloc_size = 147639500, real_size = 31871536 2024/11/23-11:39:18.514823 139957416588864 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258087.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932310 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0299 GB avg-val = 115.96 avg-zval = 32.03 2024/11/23-11:39:18.515637 139957416588864 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258087.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:39:19.348304 139957416588864 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258087.sst): second pass time = 1.31's, 82.562'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.288'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3466.542'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.330'MB/sec write SST data time = 0.13's, 285.126'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34371.939'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.562'MB/sec zip pipeline throughput = 82.562'MB/sec entries = 932310 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.96 avg-zval = 32.03 usrkeys = 932310 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.96 avg-zval = 32.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932310 bytes = 3379632 } seq expand size = 6526170 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.5130 value = 3.6201 dict = 1.00 all = 3.7178 } Zip/UnZip{ index = 0.0951 value = 0.2762 dict = 1.00 all = 0.2690 } ---------------------------- total value len = 10.326768 GB avg = 0.160 KB (by entry num) total key len = 1.783440 GB avg = 0.028 KB total ukey len = 1.242606 GB avg = 0.016 KB total ukey num = 0.077786828 Billion total entry num = 0.064594557 Billion write speed all = 1.229820677 MB/sec (with seq num) write speed all = 1.177342822 MB/sec (without seq num) 2024/11/23-11:39:19.393268 139957416588864 [compaction_job.cc:1948] [default] [JOB 25348] Generated table #258087: 932310 keys, 36101856 bytes, temperature: kWarm 2024/11/23-11:39:19.393336 139957416588864 EVENT_LOG_v1 {"time_micros": 1732333159393301, "cf_name": "default", "job": 25348, "event": "table_file_creation", "file_number": 258087, "file_size": 36101856, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 3955702836, "table_properties": {"data_size": 29864672, "index_size": 1773632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26104680, "raw_average_key_size": 28, "raw_value_size": 108113192, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932310, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732333156, "oldest_key_time": 0, "file_creation_time": 1732333156, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 258087, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:39:19.429567 139957416588864 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258088.sst) /dev/shm/ToplingTemp/Topling-38898-vKDbOr 2024/11/23-11:39:20.455173 139957416588864 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258088.sst): first pass time = 1.03's, 130.872'MB/sec 2024/11/23-11:39:21.779696 139957416588864 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258088.sst): old prealloc_size = 147639500, real_size = 33147584 2024/11/23-11:39:21.780252 139957416588864 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258088.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928196 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.60 avg-zval = 33.55 2024/11/23-11:39:21.781060 139957416588864 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258088.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:39:22.633339 139957416588864 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258088.sst): second pass time = 1.32's, 81.757'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.774'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3311.118'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.13's, 292.366'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35826.404'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.757'MB/sec zip pipeline throughput = 81.757'MB/sec entries = 928196 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.60 avg-zval = 33.55 usrkeys = 928196 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.60 avg-zval = 33.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928196 bytes = 3364720 } seq expand size = 6497372 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4444 value = 3.4758 dict = 1.00 all = 3.5920 } Zip/UnZip{ index = 0.0957 value = 0.2877 dict = 1.00 all = 0.2784 } ---------------------------- total value len = 10.441494 GB avg = 0.159 KB (by entry num) total key len = 1.809430 GB avg = 0.028 KB total ukey len = 1.261170 GB avg = 0.016 KB total ukey num = 0.078715024 Billion total entry num = 0.065522753 Billion write speed all = 1.243695734 MB/sec (with seq num) write speed all = 1.190481547 MB/sec (without seq num) 2024/11/23-11:39:22.680077 139957416588864 [compaction_job.cc:1948] [default] [JOB 25348] Generated table #258088: 928196 keys, 37365868 bytes, temperature: kWarm 2024/11/23-11:39:22.680144 139957416588864 EVENT_LOG_v1 {"time_micros": 1732333162680109, "cf_name": "default", "job": 25348, "event": "table_file_creation", "file_number": 258088, "file_size": 37365868, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 3955702851, "table_properties": {"data_size": 31137968, "index_size": 1777408, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25989488, "raw_average_key_size": 28, "raw_value_size": 108228260, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928196, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732333159, "oldest_key_time": 0, "file_creation_time": 1732333159, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 258088, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:39:22.715995 139957416588864 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258089.sst) /dev/shm/ToplingTemp/Topling-38898-zjPjgw 2024/11/23-11:39:22.917432 139957416588864 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258089.sst): first pass time = 0.20's, 250.486'MB/sec 2024/11/23-11:39:23.411888 139957416588864 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258089.sst): old prealloc_size = 147639500, real_size = 13508224 2024/11/23-11:39:23.412153 139957416588864 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258089.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 345891 prefix = 4 raw-key = 0.0055 GB zip-key = 0.0007 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0408 GB zip-val = 0.0128 GB avg-val = 117.84 avg-zval = 36.88 2024/11/23-11:39:23.412488 139957416588864 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258089.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0008 GB, waited 0.000 sec, Key+Value = 0.050 GB 2024/11/23-11:39:23.727018 139957416588864 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00/258089.sst): second pass time = 0.49's, 82.550'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.32's, 47.625'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2750.292'MB/sec (index lex order gen) rebuild zvType time = 0.27's, 0.324'MB/sec write SST data time = 0.05's, 314.256'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.415'MB, throughput = 14495.233'MB/sec zip my value time = 0.49's, unzip length = 0.041'GB zip my value throughput = 82.550'MB/sec zip pipeline throughput = 82.550'MB/sec entries = 345891 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.84 avg-zval = 36.88 usrkeys = 345891 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.84 avg-zval = 36.88 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 345891 bytes = 1167392 } seq expand size = 2421237 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0069 GB value = 0.0408 GB dict = 0.42 MB all = 0.0504 GB } __ZipSize{ index = 0.0007 GB value = 0.0128 GB dict = 0.42 MB all = 0.0150 GB } UnZip/Zip{ index = 10.4254 value = 3.1950 dict = 1.00 all = 3.3617 } Zip/UnZip{ index = 0.0959 value = 0.3130 dict = 1.00 all = 0.2975 } ---------------------------- total value len = 10.484677 GB avg = 0.159 KB (by entry num) total key len = 1.819115 GB avg = 0.028 KB total ukey len = 1.268088 GB avg = 0.016 KB total ukey num = 0.079060915 Billion total entry num = 0.065868644 Billion write speed all = 1.248924108 MB/sec (with seq num) write speed all = 1.195434945 MB/sec (without seq num) 2024/11/23-11:39:23.751460 139957416588864 [compaction_job.cc:1948] [default] [JOB 25348] Generated table #258089: 345891 keys, 15006417 bytes, temperature: kWarm 2024/11/23-11:39:23.751526 139957416588864 EVENT_LOG_v1 {"time_micros": 1732333163751488, "cf_name": "default", "job": 25348, "event": "table_file_creation", "file_number": 258089, "file_size": 15006417, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3884482859, "largest_seqno": 3955702868, "table_properties": {"data_size": 12758000, "index_size": 663552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 9684948, "raw_average_key_size": 28, "raw_value_size": 40761402, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 345891, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732333162, "oldest_key_time": 0, "file_creation_time": 1732333162, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 258089, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:39:23.767325 139957416588864 [compaction_job.cc:703] job-25348: subcompact[0], size: 0.527857 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25348/att-00 [258075,258076,258077,258078,258079,258080,258081,258082,258083,258084,258085,258086,258087,258088,258089] 2024/11/23-11:39:23.852844 139957416588864 (Original Log Time 2024/11/23-11:38:37.695373) EVENT_LOG_v1 {"time_micros": 1732333117695347, "job": 25348, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [257999], "files_L3": [258056, 258057, 257956, 257957, 257958, 257959, 257960, 257961, 257962, 257964, 257965, 257966, 257967, 257968, 258058], "score": 0.997644, "input_data_size": 552380525, "oldest_snapshot_seqno": 3956107470} 2024/11/23-11:39:23.852848 139957416588864 (Original Log Time 2024/11/23-11:39:23.851661) [compaction_job.cc:2022] [default] [JOB 25348] Compacted 1@2 + 15@3 files to L3 => 527857034 bytes 2024/11/23-11:39:23.852849 139957416588864 (Original Log Time 2024/11/23-11:39:23.852779) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 12.0 rd, 11.5 wr, level 3, files in(1, 15) out(15 +0 blob) MB in(27.1, 499.7 +0.0 blob) out(503.4 +0.0 blob), read-write-amplify(38.0) write-amplify(18.6) OK, records in: 13561505, records dropped: 51025 output_compression: Snappy 2024/11/23-11:39:23.852855 139957416588864 (Original Log Time 2024/11/23-11:39:23.852824) EVENT_LOG_v1 {"time_micros": 1732333163852804, "cf": "default", "job": 25348, "event": "compaction_finished", "compaction_time_micros": 46071960, "compaction_time_cpu_micros": 19872577, "output_level": 3, "num_output_files": 15, "total_output_size": 527857034, "num_input_records": 13561505, "num_output_records": 13510480, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 359122569, "file_range_sync_nanos": 0, "file_fsync_nanos": 10576, "file_prepare_write_nanos": 41481, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/23-11:39:23.995756 139957416588864 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732333117,"executesMs":46305,"compactionJobId":25348,"attempt":0,"compactionInputRawBytes":1936741423,"compactionInputZipBytes":552380525,"compactionOutputRawBytes":0,"compactionOutputZipBytes":527857034,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-92992e3accb003df05a2f4c200da3351-b474c52807c2c6e3-00"}}, response = 2024/11/23-11:39:23.995840 139957416588864 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-25348/att-00: olev 3, work 46.305 s, result 7.730 ms, install 1.209 ms, input{raw 1.804 GiB zip 526.791 MiB}