2024/11/24-03:47:39.788344 139957315876416 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00: fork to child time = 0.008193 sec 2024/11/24-03:47:39.789027 139957315876416 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00 2024/11/24-03:47:39.797308 139957315876416 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00 2024/11/24-03:47:39.798146 139957315876416 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-03:47:39.798330 139957315876416 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-03:47:39.853397 139957315876416 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41032/att-00/MANIFEST-206347 2024/11/24-03:47:39.869432 139957315876416 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41032/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 290848, last_sequence is 4424771323, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 290707 2024/11/24-03:47:39.869439 139957315876416 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-03:47:39.869580 139957315876416 [version_set.cc:5721] Creating manifest 290848 2024/11/24-03:47:39.903885 139957315876416 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-03:47:39.923612 139957315876416 [compaction_job.cc:2437] [default] [JOB 41032] Compacting 1@2 + 16@3 files to L3, score 0.62, subcompactions 1 : 1 2024/11/24-03:47:39.923626 139957315876416 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 30729 Base level 2, inputs: [290846(27MB)], [290761(35MB) 290762(35MB) 290763(33MB) 290764(34MB) 290765(35MB) 290778(35MB) 290779(33MB) 290780(35MB) 290781(35MB) 290782(34MB) 290783(33MB) 290784(35MB) 290785(19MB) 290786(33MB) 290787(35MB) 290788(35MB)] 2024/11/24-03:47:39.930311 139957315876416 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-03:47:39.930408 139957315876416 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-03:47:39.930487 139957315876416 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-03:47:39.930492 139957315876416 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-03:47:39.930505 139957315876416 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-03:47:40.485240 139957315876416 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290849.sst) /dev/shm/ToplingTemp/Topling-261850-zUBcp5 2024/11/24-03:47:41.079372 139957315876416 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290849.sst): first pass time = 0.59's, 225.922'MB/sec 2024/11/24-03:47:43.436667 139957315876416 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290849.sst): old prealloc_size = 147639500, real_size = 33384128 2024/11/24-03:47:43.437249 139957315876416 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290849.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926894 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.80 avg-zval = 33.86 2024/11/24-03:47:43.438041 139957315876416 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290849.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:47:45.047871 139957315876416 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290849.sst): second pass time = 2.36's, 45.945'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.61's, 23.264'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3129.614'MB/sec (index lex order gen) rebuild zvType time = 1.30's, 0.178'MB/sec write SST data time = 0.31's, 121.671'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34948.486'MB/sec zip my value time = 2.36's, unzip length = 0.108'GB zip my value throughput = 45.945'MB/sec zip pipeline throughput = 91.889'MB/sec entries = 926894 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.80 avg-zval = 33.86 usrkeys = 926894 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.80 avg-zval = 33.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926894 bytes = 3244144 } seq expand size = 6488258 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5130 value = 3.4491 dict = 1.00 all = 3.5808 } Zip/UnZip{ index = 0.0951 value = 0.2899 dict = 1.00 all = 0.2793 } ---------------------------- total value len = 2251.440326 GB avg = 0.200 KB (by entry num) total key len = 306.908916 GB avg = 0.027 KB total ukey len = 237.373265 GB avg = 0.015 KB total ukey num = 15.739588057 Billion total entry num = 11.280902005 Billion write speed all = 37.648897354 MB/sec (with seq num) write speed all = 36.320811212 MB/sec (without seq num) 2024/11/24-03:47:45.108727 139957315876416 [compaction_job.cc:1948] [default] [JOB 41032] Generated table #290849: 926894 keys, 37482432 bytes, temperature: kWarm 2024/11/24-03:47:45.108839 139957315876416 EVENT_LOG_v1 {"time_micros": 1732391265108769, "cf_name": "default", "job": 41032, "event": "table_file_creation", "file_number": 290849, "file_size": 37482432, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4259662645, "largest_seqno": 4424754862, "table_properties": {"data_size": 31388912, "index_size": 1763328, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25953032, "raw_average_key_size": 28, "raw_value_size": 108264780, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926894, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391260, "oldest_key_time": 0, "file_creation_time": 1732391260, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290849, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:47:45.185458 139957315876416 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290850.sst) /dev/shm/ToplingTemp/Topling-261850-AxQxL3 2024/11/24-03:47:46.066701 139957315876416 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290850.sst): first pass time = 0.88's, 152.315'MB/sec 2024/11/24-03:47:48.419359 139957315876416 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290850.sst): old prealloc_size = 147639500, real_size = 32752032 2024/11/24-03:47:48.420072 139957315876416 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290850.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927026 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0307 GB avg-val = 116.78 avg-zval = 33.17 2024/11/24-03:47:48.420932 139957315876416 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290850.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:47:50.252743 139957315876416 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290850.sst): second pass time = 2.35's, 46.033'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.83's, 20.100'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2551.636'MB/sec (index lex order gen) rebuild zvType time = 1.59's, 0.146'MB/sec write SST data time = 0.24's, 153.496'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33157.455'MB/sec zip my value time = 2.35's, unzip length = 0.108'GB zip my value throughput = 46.033'MB/sec zip pipeline throughput = 121.834'MB/sec entries = 927026 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.78 avg-zval = 33.17 usrkeys = 927026 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.78 avg-zval = 33.17 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927026 bytes = 3244608 } seq expand size = 6489182 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4448 value = 3.5213 dict = 1.00 all = 3.6422 } Zip/UnZip{ index = 0.0957 value = 0.2840 dict = 1.00 all = 0.2746 } ---------------------------- total value len = 2251.555077 GB avg = 0.200 KB (by entry num) total key len = 306.934872 GB avg = 0.027 KB total ukey len = 237.391806 GB avg = 0.015 KB total ukey num = 15.740515083 Billion total entry num = 11.281829031 Billion write speed all = 37.648084329 MB/sec (with seq num) write speed all = 36.319990776 MB/sec (without seq num) 2024/11/24-03:47:50.301197 139957315876416 [compaction_job.cc:1948] [default] [JOB 41032] Generated table #290850: 927026 keys, 36851206 bytes, temperature: kWarm 2024/11/24-03:47:50.301283 139957315876416 EVENT_LOG_v1 {"time_micros": 1732391270301229, "cf_name": "default", "job": 41032, "event": "table_file_creation", "file_number": 290850, "file_size": 36851206, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178503, "largest_seqno": 4424754696, "table_properties": {"data_size": 30744976, "index_size": 1775104, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25956728, "raw_average_key_size": 28, "raw_value_size": 108261092, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927026, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391265, "oldest_key_time": 0, "file_creation_time": 1732391265, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290850, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:47:50.359017 139957315876416 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290851.sst) /dev/shm/ToplingTemp/Topling-261850-tYb5X2 2024/11/24-03:47:51.240951 139957315876416 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290851.sst): first pass time = 0.88's, 152.192'MB/sec 2024/11/24-03:47:54.609320 139957315876416 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290851.sst): old prealloc_size = 147639500, real_size = 31155392 2024/11/24-03:47:54.609877 139957315876416 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290851.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923986 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0291 GB avg-val = 117.26 avg-zval = 31.54 2024/11/24-03:47:54.610714 139957315876416 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290851.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:47:55.439379 139957315876416 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290851.sst): second pass time = 3.37's, 32.178'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.322'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3338.495'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.332'MB/sec write SST data time = 0.13's, 262.678'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 34376.787'MB/sec zip my value time = 3.37's, unzip length = 0.108'GB zip my value throughput = 32.178'MB/sec zip pipeline throughput = 120.983'MB/sec entries = 923986 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.26 avg-zval = 31.54 usrkeys = 923986 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.26 avg-zval = 31.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923986 bytes = 3118464 } seq expand size = 6467902 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.3750 value = 3.7177 dict = 1.00 all = 3.8207 } Zip/UnZip{ index = 0.0964 value = 0.2690 dict = 1.00 all = 0.2617 } ---------------------------- total value len = 2251.669891 GB avg = 0.200 KB (by entry num) total key len = 306.960744 GB avg = 0.027 KB total ukey len = 237.410285 GB avg = 0.015 KB total ukey num = 15.741439069 Billion total entry num = 11.282753017 Billion write speed all = 37.647281224 MB/sec (with seq num) write speed all = 36.319180262 MB/sec (without seq num) 2024/11/24-03:47:55.488560 139957315876416 [compaction_job.cc:1948] [default] [JOB 41032] Generated table #290851: 923986 keys, 35129114 bytes, temperature: kWarm 2024/11/24-03:47:55.488614 139957315876416 EVENT_LOG_v1 {"time_micros": 1732391275488584, "cf_name": "default", "job": 41032, "event": "table_file_creation", "file_number": 290851, "file_size": 35129114, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4299864800, "largest_seqno": 4424754707, "table_properties": {"data_size": 29143024, "index_size": 1781184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25871608, "raw_average_key_size": 28, "raw_value_size": 108346212, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923986, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391270, "oldest_key_time": 0, "file_creation_time": 1732391270, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290851, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:47:55.537897 139957315876416 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290852.sst) /dev/shm/ToplingTemp/Topling-261850-8i2dS4 2024/11/24-03:47:56.389599 139957315876416 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290852.sst): first pass time = 0.85's, 157.593'MB/sec 2024/11/24-03:47:57.993044 139957315876416 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290852.sst): old prealloc_size = 147639500, real_size = 32506720 2024/11/24-03:47:57.993613 139957315876416 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290852.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922165 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0305 GB avg-val = 117.55 avg-zval = 33.10 2024/11/24-03:47:57.994389 139957315876416 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290852.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:47:58.824353 139957315876416 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290852.sst): second pass time = 1.60's, 67.638'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 44.020'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3212.195'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.329'MB/sec write SST data time = 0.13's, 281.554'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34833.607'MB/sec zip my value time = 1.60's, unzip length = 0.108'GB zip my value throughput = 67.638'MB/sec zip pipeline throughput = 115.779'MB/sec entries = 922165 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.55 avg-zval = 33.10 usrkeys = 922165 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.55 avg-zval = 33.10 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922165 bytes = 3227600 } seq expand size = 6455155 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5143 value = 3.5515 dict = 1.00 all = 3.6679 } Zip/UnZip{ index = 0.0951 value = 0.2816 dict = 1.00 all = 0.2726 } ---------------------------- total value len = 2251.784743 GB avg = 0.200 KB (by entry num) total key len = 306.986564 GB avg = 0.027 KB total ukey len = 237.428729 GB avg = 0.015 KB total ukey num = 15.742361234 Billion total entry num = 11.283675182 Billion write speed all = 37.647475999 MB/sec (with seq num) write speed all = 36.319332638 MB/sec (without seq num) 2024/11/24-03:47:58.869356 139957315876416 [compaction_job.cc:1948] [default] [JOB 41032] Generated table #290852: 922165 keys, 36592944 bytes, temperature: kWarm 2024/11/24-03:47:58.869416 139957315876416 EVENT_LOG_v1 {"time_micros": 1732391278869382, "cf_name": "default", "job": 41032, "event": "table_file_creation", "file_number": 290852, "file_size": 36592944, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178518, "largest_seqno": 4424753742, "table_properties": {"data_size": 30521872, "index_size": 1754112, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25820620, "raw_average_key_size": 28, "raw_value_size": 108397118, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922165, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391275, "oldest_key_time": 0, "file_creation_time": 1732391275, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290852, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:47:58.907051 139957315876416 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290853.sst) /dev/shm/ToplingTemp/Topling-261850-Ra7N35 2024/11/24-03:47:59.821505 139957315876416 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290853.sst): first pass time = 0.91's, 146.781'MB/sec 2024/11/24-03:48:01.464875 139957315876416 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290853.sst): old prealloc_size = 147639500, real_size = 33448992 2024/11/24-03:48:01.465433 139957315876416 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290853.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926926 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.80 avg-zval = 33.92 2024/11/24-03:48:01.466207 139957315876416 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290853.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:48:02.328550 139957315876416 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290853.sst): second pass time = 1.64's, 65.911'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.476'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3301.856'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 285.003'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 26954.066'MB/sec zip my value time = 1.64's, unzip length = 0.108'GB zip my value throughput = 65.911'MB/sec zip pipeline throughput = 128.065'MB/sec entries = 926926 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.80 avg-zval = 33.92 usrkeys = 926926 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.80 avg-zval = 33.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926926 bytes = 3244256 } seq expand size = 6488482 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4436 value = 3.4433 dict = 1.00 all = 3.5746 } Zip/UnZip{ index = 0.0958 value = 0.2904 dict = 1.00 all = 0.2798 } ---------------------------- total value len = 2251.899495 GB avg = 0.200 KB (by entry num) total key len = 307.012518 GB avg = 0.027 KB total ukey len = 237.447267 GB avg = 0.015 KB total ukey num = 15.743288160 Billion total entry num = 11.284602108 Billion write speed all = 37.647605205 MB/sec (with seq num) write speed all = 36.319421218 MB/sec (without seq num) 2024/11/24-03:48:02.379772 139957315876416 [compaction_job.cc:1948] [default] [JOB 41032] Generated table #290853: 926926 keys, 37547636 bytes, temperature: kWarm 2024/11/24-03:48:02.379834 139957315876416 EVENT_LOG_v1 {"time_micros": 1732391282379801, "cf_name": "default", "job": 41032, "event": "table_file_creation", "file_number": 290853, "file_size": 37547636, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178530, "largest_seqno": 4424754717, "table_properties": {"data_size": 31442000, "index_size": 1775104, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25953928, "raw_average_key_size": 28, "raw_value_size": 108263848, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926926, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391278, "oldest_key_time": 0, "file_creation_time": 1732391278, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290853, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:48:02.423433 139957315876416 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290854.sst) /dev/shm/ToplingTemp/Topling-261850-aepev6 2024/11/24-03:48:03.309680 139957315876416 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290854.sst): first pass time = 0.89's, 151.451'MB/sec 2024/11/24-03:48:04.899963 139957315876416 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290854.sst): old prealloc_size = 147639500, real_size = 32325216 2024/11/24-03:48:04.900493 139957315876416 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290854.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926887 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0303 GB avg-val = 116.80 avg-zval = 32.72 2024/11/24-03:48:04.901286 139957315876416 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290854.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:48:05.739371 139957315876416 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290854.sst): second pass time = 1.59's, 68.109'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.394'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3533.052'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 281.017'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 28236.407'MB/sec zip my value time = 1.59's, unzip length = 0.108'GB zip my value throughput = 68.109'MB/sec zip pipeline throughput = 124.200'MB/sec entries = 926887 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.80 avg-zval = 32.72 usrkeys = 926887 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.80 avg-zval = 32.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926887 bytes = 3244112 } seq expand size = 6488209 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5125 value = 3.5696 dict = 1.00 all = 3.6849 } Zip/UnZip{ index = 0.0951 value = 0.2801 dict = 1.00 all = 0.2714 } ---------------------------- total value len = 2252.014248 GB avg = 0.200 KB (by entry num) total key len = 307.038471 GB avg = 0.027 KB total ukey len = 237.465805 GB avg = 0.015 KB total ukey num = 15.744215047 Billion total entry num = 11.285528995 Billion write speed all = 37.647786110 MB/sec (with seq num) write speed all = 36.319559681 MB/sec (without seq num) 2024/11/24-03:48:05.815593 139957315876416 [compaction_job.cc:1948] [default] [JOB 41032] Generated table #290854: 926887 keys, 36423652 bytes, temperature: kWarm 2024/11/24-03:48:05.815640 139957315876416 EVENT_LOG_v1 {"time_micros": 1732391285815613, "cf_name": "default", "job": 41032, "event": "table_file_creation", "file_number": 290854, "file_size": 36423652, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178548, "largest_seqno": 4424754732, "table_properties": {"data_size": 30329936, "index_size": 1763392, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25952836, "raw_average_key_size": 28, "raw_value_size": 108264978, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926887, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391282, "oldest_key_time": 0, "file_creation_time": 1732391282, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290854, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:48:05.853105 139957315876416 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290855.sst) /dev/shm/ToplingTemp/Topling-261850-1I9ux2 2024/11/24-03:48:06.708699 139957315876416 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290855.sst): first pass time = 0.86's, 156.878'MB/sec 2024/11/24-03:48:08.309279 139957315876416 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290855.sst): old prealloc_size = 147639500, real_size = 31260352 2024/11/24-03:48:08.309777 139957315876416 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290855.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921635 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0293 GB avg-val = 117.63 avg-zval = 31.75 2024/11/24-03:48:08.310537 139957315876416 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290855.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:48:09.127076 139957315876416 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290855.sst): second pass time = 1.60's, 67.762'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.219'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3683.331'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 271.157'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 32710.193'MB/sec zip my value time = 1.60's, unzip length = 0.108'GB zip my value throughput = 67.762'MB/sec zip pipeline throughput = 135.448'MB/sec entries = 921635 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.63 avg-zval = 31.75 usrkeys = 921635 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.63 avg-zval = 31.75 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921635 bytes = 3225744 } seq expand size = 6451445 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.08 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4462 value = 3.7045 dict = 1.00 all = 3.7975 } Zip/UnZip{ index = 0.0957 value = 0.2699 dict = 1.00 all = 0.2633 } ---------------------------- total value len = 2252.129112 GB avg = 0.200 KB (by entry num) total key len = 307.064277 GB avg = 0.027 KB total ukey len = 237.484238 GB avg = 0.015 KB total ukey num = 15.745136682 Billion total entry num = 11.286450630 Billion write speed all = 37.647979251 MB/sec (with seq num) write speed all = 36.319710552 MB/sec (without seq num) 2024/11/24-03:48:09.204547 139957315876416 [compaction_job.cc:1948] [default] [JOB 41032] Generated table #290855: 921635 keys, 35343320 bytes, temperature: kWarm 2024/11/24-03:48:09.204595 139957315876416 EVENT_LOG_v1 {"time_micros": 1732391289204567, "cf_name": "default", "job": 41032, "event": "table_file_creation", "file_number": 290855, "file_size": 35343320, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4286327441, "largest_seqno": 4424754747, "table_properties": {"data_size": 29265200, "index_size": 1764544, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25805780, "raw_average_key_size": 28, "raw_value_size": 108411950, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921635, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391285, "oldest_key_time": 0, "file_creation_time": 1732391285, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290855, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:48:09.251015 139957315876416 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290856.sst) /dev/shm/ToplingTemp/Topling-261850-KjmpC3 2024/11/24-03:48:10.111872 139957315876416 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290856.sst): first pass time = 0.86's, 155.918'MB/sec 2024/11/24-03:48:11.725481 139957315876416 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290856.sst): old prealloc_size = 147639500, real_size = 32907920 2024/11/24-03:48:11.726070 139957315876416 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290856.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924480 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 117.18 avg-zval = 33.44 2024/11/24-03:48:11.726842 139957315876416 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290856.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:48:12.571926 139957315876416 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290856.sst): second pass time = 1.61's, 67.172'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.713'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3096.645'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 288.277'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 36477.312'MB/sec zip my value time = 1.61's, unzip length = 0.108'GB zip my value throughput = 67.172'MB/sec zip pipeline throughput = 126.701'MB/sec entries = 924480 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.18 avg-zval = 33.44 usrkeys = 924480 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.18 avg-zval = 33.44 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924480 bytes = 3235696 } seq expand size = 6471360 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5135 value = 3.5039 dict = 1.00 all = 3.6276 } Zip/UnZip{ index = 0.0951 value = 0.2854 dict = 1.00 all = 0.2757 } ---------------------------- total value len = 2252.243916 GB avg = 0.200 KB (by entry num) total key len = 307.090162 GB avg = 0.027 KB total ukey len = 237.502727 GB avg = 0.015 KB total ukey num = 15.746061162 Billion total entry num = 11.287375110 Billion write speed all = 37.648141032 MB/sec (with seq num) write speed all = 36.319830848 MB/sec (without seq num) 2024/11/24-03:48:12.620949 139957315876416 [compaction_job.cc:1948] [default] [JOB 41032] Generated table #290856: 924480 keys, 36999512 bytes, temperature: kWarm 2024/11/24-03:48:12.621022 139957315876416 EVENT_LOG_v1 {"time_micros": 1732391292620987, "cf_name": "default", "job": 41032, "event": "table_file_creation", "file_number": 290856, "file_size": 36999512, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178559, "largest_seqno": 4424750869, "table_properties": {"data_size": 30917952, "index_size": 1758656, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25885440, "raw_average_key_size": 28, "raw_value_size": 108332348, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924480, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391289, "oldest_key_time": 0, "file_creation_time": 1732391289, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290856, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:48:12.659319 139957315876416 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290857.sst) /dev/shm/ToplingTemp/Topling-261850-kr4l82 2024/11/24-03:48:13.552552 139957315876416 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290857.sst): first pass time = 0.89's, 150.267'MB/sec 2024/11/24-03:48:15.134532 139957315876416 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290857.sst): old prealloc_size = 147639500, real_size = 33432928 2024/11/24-03:48:15.135083 139957315876416 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290857.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926941 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.80 avg-zval = 33.90 2024/11/24-03:48:15.135857 139957315876416 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290857.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:48:15.995299 139957315876416 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290857.sst): second pass time = 1.58's, 68.486'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.604'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3352.287'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.316'MB/sec write SST data time = 0.13's, 296.569'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 22523.998'MB/sec zip my value time = 1.58's, unzip length = 0.108'GB zip my value throughput = 68.486'MB/sec zip pipeline throughput = 104.680'MB/sec entries = 926941 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.80 avg-zval = 33.90 usrkeys = 926941 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.80 avg-zval = 33.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926941 bytes = 3244304 } seq expand size = 6488587 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4453 value = 3.4450 dict = 1.00 all = 3.5761 } Zip/UnZip{ index = 0.0957 value = 0.2903 dict = 1.00 all = 0.2796 } ---------------------------- total value len = 2252.358668 GB avg = 0.200 KB (by entry num) total key len = 307.116117 GB avg = 0.027 KB total ukey len = 237.521266 GB avg = 0.015 KB total ukey num = 15.746988103 Billion total entry num = 11.288302051 Billion write speed all = 37.648314939 MB/sec (with seq num) write speed all = 36.319962565 MB/sec (without seq num) 2024/11/24-03:48:16.044154 139957315876416 [compaction_job.cc:1948] [default] [JOB 41032] Generated table #290857: 926941 keys, 37531640 bytes, temperature: kWarm 2024/11/24-03:48:16.044223 139957315876416 EVENT_LOG_v1 {"time_micros": 1732391296044188, "cf_name": "default", "job": 41032, "event": "table_file_creation", "file_number": 290857, "file_size": 37531640, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178575, "largest_seqno": 4424754764, "table_properties": {"data_size": 31426192, "index_size": 1774848, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25954348, "raw_average_key_size": 28, "raw_value_size": 108263486, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926941, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391292, "oldest_key_time": 0, "file_creation_time": 1732391292, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290857, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:48:16.081880 139957315876416 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290858.sst) /dev/shm/ToplingTemp/Topling-261850-qRPiF5 2024/11/24-03:48:17.004173 139957315876416 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290858.sst): first pass time = 0.92's, 145.532'MB/sec 2024/11/24-03:48:18.446565 139957315876416 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290858.sst): old prealloc_size = 147639500, real_size = 31635376 2024/11/24-03:48:18.447090 139957315876416 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290858.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926804 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0296 GB avg-val = 116.82 avg-zval = 31.98 2024/11/24-03:48:18.448938 139957315876416 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290858.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:48:19.282314 139957315876416 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290858.sst): second pass time = 1.44's, 75.139'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.758'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3514.604'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.328'MB/sec write SST data time = 0.13's, 277.604'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35757.887'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.139'MB/sec zip pipeline throughput = 75.139'MB/sec entries = 926804 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.82 avg-zval = 31.98 usrkeys = 926804 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.82 avg-zval = 31.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926804 bytes = 3243824 } seq expand size = 6487628 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5112 value = 3.6527 dict = 1.00 all = 3.7561 } Zip/UnZip{ index = 0.0951 value = 0.2738 dict = 1.00 all = 0.2662 } ---------------------------- total value len = 2252.473422 GB avg = 0.200 KB (by entry num) total key len = 307.142067 GB avg = 0.027 KB total ukey len = 237.539802 GB avg = 0.015 KB total ukey num = 15.747914907 Billion total entry num = 11.289228855 Billion write speed all = 37.648564322 MB/sec (with seq num) write speed all = 36.320167115 MB/sec (without seq num) 2024/11/24-03:48:19.327567 139957315876416 [compaction_job.cc:1948] [default] [JOB 41032] Generated table #290858: 926804 keys, 35733538 bytes, temperature: kWarm 2024/11/24-03:48:19.327636 139957315876416 EVENT_LOG_v1 {"time_micros": 1732391299327601, "cf_name": "default", "job": 41032, "event": "table_file_creation", "file_number": 290858, "file_size": 35733538, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178589, "largest_seqno": 4424754774, "table_properties": {"data_size": 29640032, "index_size": 1763456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25950512, "raw_average_key_size": 28, "raw_value_size": 108267272, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926804, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391296, "oldest_key_time": 0, "file_creation_time": 1732391296, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290858, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:48:19.371850 139957315876416 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290859.sst) /dev/shm/ToplingTemp/Topling-261850-2IpJV2 2024/11/24-03:48:20.303094 139957315876416 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290859.sst): first pass time = 0.93's, 144.134'MB/sec 2024/11/24-03:48:21.752456 139957315876416 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290859.sst): old prealloc_size = 147639500, real_size = 31399776 2024/11/24-03:48:21.753022 139957315876416 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290859.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919418 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 117.98 avg-zval = 31.99 2024/11/24-03:48:21.754850 139957315876416 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290859.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:48:22.572271 139957315876416 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290859.sst): second pass time = 1.45's, 74.925'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.276'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3254.350'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 271.661'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 28675.106'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.925'MB/sec zip pipeline throughput = 77.066'MB/sec entries = 919418 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.98 avg-zval = 31.99 usrkeys = 919418 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.98 avg-zval = 31.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919418 bytes = 3217984 } seq expand size = 6435926 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4460 value = 3.6884 dict = 1.00 all = 3.7832 } Zip/UnZip{ index = 0.0957 value = 0.2711 dict = 1.00 all = 0.2643 } ---------------------------- total value len = 2252.588332 GB avg = 0.200 KB (by entry num) total key len = 307.167811 GB avg = 0.027 KB total ukey len = 237.558191 GB avg = 0.015 KB total ukey num = 15.748834325 Billion total entry num = 11.290148273 Billion write speed all = 37.648811291 MB/sec (with seq num) write speed all = 36.320370180 MB/sec (without seq num) 2024/11/24-03:48:22.620885 139957315876416 [compaction_job.cc:1948] [default] [JOB 41032] Generated table #290859: 919418 keys, 35476954 bytes, temperature: kWarm 2024/11/24-03:48:22.620958 139957315876416 EVENT_LOG_v1 {"time_micros": 1732391302620923, "cf_name": "default", "job": 41032, "event": "table_file_creation", "file_number": 290859, "file_size": 35476954, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4273060473, "largest_seqno": 4424754792, "table_properties": {"data_size": 29409424, "index_size": 1760320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25743704, "raw_average_key_size": 28, "raw_value_size": 108474080, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 919418, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391299, "oldest_key_time": 0, "file_creation_time": 1732391299, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290859, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:48:22.657465 139957315876416 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290860.sst) /dev/shm/ToplingTemp/Topling-261850-vD54v6 2024/11/24-03:48:23.517728 139957315876416 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290860.sst): first pass time = 0.86's, 156.026'MB/sec 2024/11/24-03:48:24.993256 139957315876416 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290860.sst): old prealloc_size = 147639500, real_size = 33122592 2024/11/24-03:48:24.993770 139957315876416 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290860.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926756 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 116.83 avg-zval = 33.59 2024/11/24-03:48:24.994591 139957315876416 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290860.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:48:25.859114 139957315876416 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290860.sst): second pass time = 1.47's, 73.413'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.988'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3560.495'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.13's, 288.281'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31317.482'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.413'MB/sec zip pipeline throughput = 73.842'MB/sec entries = 926756 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.83 avg-zval = 33.59 usrkeys = 926756 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.83 avg-zval = 33.59 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926756 bytes = 3243664 } seq expand size = 6487292 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5114 value = 3.4782 dict = 1.00 all = 3.6060 } Zip/UnZip{ index = 0.0951 value = 0.2875 dict = 1.00 all = 0.2773 } ---------------------------- total value len = 2252.703088 GB avg = 0.200 KB (by entry num) total key len = 307.193760 GB avg = 0.027 KB total ukey len = 237.576726 GB avg = 0.015 KB total ukey num = 15.749761081 Billion total entry num = 11.291075029 Billion write speed all = 37.649060716 MB/sec (with seq num) write speed all = 36.320574782 MB/sec (without seq num) 2024/11/24-03:48:25.908306 139957315876416 [compaction_job.cc:1948] [default] [JOB 41032] Generated table #290860: 926756 keys, 37220414 bytes, temperature: kWarm 2024/11/24-03:48:25.908375 139957315876416 EVENT_LOG_v1 {"time_micros": 1732391305908340, "cf_name": "default", "job": 41032, "event": "table_file_creation", "file_number": 290860, "file_size": 37220414, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178599, "largest_seqno": 4424754427, "table_properties": {"data_size": 31127376, "index_size": 1763328, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25949168, "raw_average_key_size": 28, "raw_value_size": 108268604, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926756, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391302, "oldest_key_time": 0, "file_creation_time": 1732391302, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290860, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:48:25.947969 139957315876416 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290861.sst) /dev/shm/ToplingTemp/Topling-261850-YumOL5 2024/11/24-03:48:26.963387 139957315876416 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290861.sst): first pass time = 1.02's, 86.322'MB/sec 2024/11/24-03:48:28.052176 139957315876416 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290861.sst): old prealloc_size = 147639500, real_size = 21088224 2024/11/24-03:48:28.052550 139957315876416 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290861.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 607997 prefix = 4 raw-key = 0.0097 GB zip-key = 0.0012 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0706 GB zip-val = 0.0198 GB avg-val = 116.16 avg-zval = 32.51 2024/11/24-03:48:28.053077 139957315876416 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290861.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0015 GB, waited 0.000 sec, Key+Value = 0.088 GB 2024/11/24-03:48:28.627055 139957315876416 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290861.sst): second pass time = 1.09's, 64.909'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.57's, 41.097'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3344.090'MB/sec (index lex order gen) rebuild zvType time = 0.49's, 0.310'MB/sec write SST data time = 0.08's, 279.820'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.709'MB, throughput = 18504.975'MB/sec zip my value time = 1.09's, unzip length = 0.071'GB zip my value throughput = 64.909'MB/sec zip pipeline throughput = 143.881'MB/sec entries = 607997 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.16 avg-zval = 32.51 usrkeys = 607997 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.16 avg-zval = 32.51 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 607997 bytes = 1976000 } seq expand size = 4255979 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0122 GB value = 0.0706 GB dict = 0.71 MB all = 0.0876 GB } __ZipSize{ index = 0.0012 GB value = 0.0198 GB dict = 0.71 MB all = 0.0236 GB } UnZip/Zip{ index = 10.3734 value = 3.5734 dict = 1.00 all = 3.7101 } Zip/UnZip{ index = 0.0964 value = 0.2798 dict = 1.00 all = 0.2695 } ---------------------------- total value len = 2252.777968 GB avg = 0.200 KB (by entry num) total key len = 307.210784 GB avg = 0.027 KB total ukey len = 237.588886 GB avg = 0.015 KB total ukey num = 15.750369078 Billion total entry num = 11.291683026 Billion write speed all = 37.648879746 MB/sec (with seq num) write speed all = 36.320376358 MB/sec (without seq num) 2024/11/24-03:48:28.659665 139957315876416 [compaction_job.cc:1948] [default] [JOB 41032] Generated table #290861: 607997 keys, 23623863 bytes, temperature: kWarm 2024/11/24-03:48:28.659733 139957315876416 EVENT_LOG_v1 {"time_micros": 1732391308659699, "cf_name": "default", "job": 41032, "event": "table_file_creation", "file_number": 290861, "file_size": 23623863, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4370686723, "largest_seqno": 4424754805, "table_properties": {"data_size": 19763792, "index_size": 1172224, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 17023916, "raw_average_key_size": 28, "raw_value_size": 70624094, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 607997, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391305, "oldest_key_time": 0, "file_creation_time": 1732391305, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290861, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:48:28.692304 139957315876416 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290862.sst) /dev/shm/ToplingTemp/Topling-261850-sJEvS3 2024/11/24-03:48:29.614016 139957315876416 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290862.sst): first pass time = 0.92's, 145.624'MB/sec 2024/11/24-03:48:31.064305 139957315876416 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290862.sst): old prealloc_size = 147639500, real_size = 31398672 2024/11/24-03:48:31.064821 139957315876416 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290862.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.99 2024/11/24-03:48:31.066634 139957315876416 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290862.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:48:31.879637 139957315876416 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290862.sst): second pass time = 1.45's, 74.839'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.512'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3568.770'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.13's, 282.365'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 35333.138'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.839'MB/sec zip pipeline throughput = 74.839'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.99 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4458 value = 3.6886 dict = 1.00 all = 3.7834 } Zip/UnZip{ index = 0.0957 value = 0.2711 dict = 1.00 all = 0.2643 } ---------------------------- total value len = 2252.892881 GB avg = 0.200 KB (by entry num) total key len = 307.236525 GB avg = 0.027 KB total ukey len = 237.607272 GB avg = 0.015 KB total ukey num = 15.751288378 Billion total entry num = 11.292602326 Billion write speed all = 37.649147333 MB/sec (with seq num) write speed all = 36.320599338 MB/sec (without seq num) 2024/11/24-03:48:31.923544 139957315876416 [compaction_job.cc:1948] [default] [JOB 41032] Generated table #290862: 919300 keys, 35475378 bytes, temperature: kWarm 2024/11/24-03:48:31.923617 139957315876416 EVENT_LOG_v1 {"time_micros": 1732391311923583, "cf_name": "default", "job": 41032, "event": "table_file_creation", "file_number": 290862, "file_size": 35475378, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178645, "largest_seqno": 4414357721, "table_properties": {"data_size": 29408512, "index_size": 1760128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391308, "oldest_key_time": 0, "file_creation_time": 1732391308, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290862, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:48:31.962150 139957315876416 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290863.sst) /dev/shm/ToplingTemp/Topling-261850-fCEui3 2024/11/24-03:48:32.497783 139957315876416 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290863.sst): first pass time = 0.54's, 250.593'MB/sec 2024/11/24-03:48:33.893497 139957315876416 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290863.sst): old prealloc_size = 147639500, real_size = 33222320 2024/11/24-03:48:33.894071 139957315876416 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290863.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926829 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 116.81 avg-zval = 33.69 2024/11/24-03:48:33.894854 139957315876416 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290863.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:48:34.859014 139957315876416 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290863.sst): second pass time = 1.39's, 77.635'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.97's, 38.655'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3197.847'MB/sec (index lex order gen) rebuild zvType time = 0.84's, 0.277'MB/sec write SST data time = 0.13's, 291.329'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 25906.537'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 77.635'MB/sec zip pipeline throughput = 77.635'MB/sec entries = 926829 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.81 avg-zval = 33.69 usrkeys = 926829 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.81 avg-zval = 33.69 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926829 bytes = 3243920 } seq expand size = 6487803 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5111 value = 3.4671 dict = 1.00 all = 3.5964 } Zip/UnZip{ index = 0.0951 value = 0.2884 dict = 1.00 all = 0.2781 } ---------------------------- total value len = 2253.007635 GB avg = 0.199 KB (by entry num) total key len = 307.262476 GB avg = 0.027 KB total ukey len = 237.625808 GB avg = 0.015 KB total ukey num = 15.752215207 Billion total entry num = 11.293529155 Billion write speed all = 37.649566952 MB/sec (with seq num) write speed all = 36.320968129 MB/sec (without seq num) 2024/11/24-03:48:34.907213 139957315876416 [compaction_job.cc:1948] [default] [JOB 41032] Generated table #290863: 926829 keys, 37320314 bytes, temperature: kWarm 2024/11/24-03:48:34.907278 139957315876416 EVENT_LOG_v1 {"time_micros": 1732391314907244, "cf_name": "default", "job": 41032, "event": "table_file_creation", "file_number": 290863, "file_size": 37320314, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178655, "largest_seqno": 4424754491, "table_properties": {"data_size": 31226912, "index_size": 1763520, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25951212, "raw_average_key_size": 28, "raw_value_size": 108266542, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926829, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391311, "oldest_key_time": 0, "file_creation_time": 1732391311, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290863, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:48:34.942139 139957315876416 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290864.sst) /dev/shm/ToplingTemp/Topling-261850-eqVjI2 2024/11/24-03:48:35.925389 139957315876416 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290864.sst): first pass time = 0.98's, 136.510'MB/sec 2024/11/24-03:48:37.308297 139957315876416 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290864.sst): old prealloc_size = 147639500, real_size = 33155936 2024/11/24-03:48:37.308841 139957315876416 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290864.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926547 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 116.86 avg-zval = 33.63 2024/11/24-03:48:37.310625 139957315876416 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290864.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:48:38.271643 139957315876416 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290864.sst): second pass time = 1.38's, 78.342'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 38.672'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3435.749'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.278'MB/sec write SST data time = 0.13's, 288.934'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32383.613'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.342'MB/sec zip pipeline throughput = 78.342'MB/sec entries = 926547 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.86 avg-zval = 33.63 usrkeys = 926547 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.86 avg-zval = 33.63 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926547 bytes = 3242928 } seq expand size = 6485829 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5125 value = 3.4746 dict = 1.00 all = 3.6028 } Zip/UnZip{ index = 0.0951 value = 0.2878 dict = 1.00 all = 0.2776 } ---------------------------- total value len = 2253.122396 GB avg = 0.199 KB (by entry num) total key len = 307.288419 GB avg = 0.027 KB total ukey len = 237.644339 GB avg = 0.015 KB total ukey num = 15.753141754 Billion total entry num = 11.294455702 Billion write speed all = 37.649746626 MB/sec (with seq num) write speed all = 36.321105478 MB/sec (without seq num) 2024/11/24-03:48:38.320177 139957315876416 [compaction_job.cc:1948] [default] [JOB 41032] Generated table #290864: 926547 keys, 37253552 bytes, temperature: kWarm 2024/11/24-03:48:38.320251 139957315876416 EVENT_LOG_v1 {"time_micros": 1732391318320215, "cf_name": "default", "job": 41032, "event": "table_file_creation", "file_number": 290864, "file_size": 37253552, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178675, "largest_seqno": 4424753971, "table_properties": {"data_size": 31161360, "index_size": 1762752, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25943316, "raw_average_key_size": 28, "raw_value_size": 108274470, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926547, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391314, "oldest_key_time": 0, "file_creation_time": 1732391314, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290864, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:48:38.367157 139957315876416 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290865.sst) /dev/shm/ToplingTemp/Topling-261850-EAO6d3 2024/11/24-03:48:38.383182 139957315876416 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290865.sst): first pass time = 0.02's, 180.500'MB/sec 2024/11/24-03:48:38.422807 139957315876416 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290865.sst): old prealloc_size = 147639500, real_size = 962080 2024/11/24-03:48:38.422852 139957315876416 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290865.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 19764 prefix = 4 raw-key = 0.0003 GB zip-key = 0.0000 GB avg-key = 16.00 avg-zkey = 1.99 raw-val = 0.0023 GB zip-val = 0.0009 GB avg-val = 118.00 avg-zval = 46.43 2024/11/24-03:48:38.422887 139957315876416 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290865.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0000 GB, waited 0.000 sec, Key+Value = 0.003 GB 2024/11/24-03:48:38.438775 139957315876416 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00/290865.sst): second pass time = 0.04's, 59.586'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.02's, 64.923'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1242.876'MB/sec (index lex order gen) rebuild zvType time = 0.01's, 0.381'MB/sec write SST data time = 0.00's, 352.107'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.023'MB, throughput = 895.058'MB/sec zip my value time = 0.04's, unzip length = 0.002'GB zip my value throughput = 59.586'MB/sec zip pipeline throughput = 59.586'MB/sec entries = 19764 avg-key = 28.00 avg-zkey = 1.99 avg-val = 118.00 avg-zval = 46.43 usrkeys = 19764 avg-key = 20.00 avg-zkey = 1.99 avg-val = 125.00 avg-zval = 46.43 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 19764 bytes = 54368 } seq expand size = 138348 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0004 GB value = 0.0023 GB dict = 0.02 MB all = 0.0029 GB } __ZipSize{ index = 0.0000 GB value = 0.0009 GB dict = 0.02 MB all = 0.0010 GB } UnZip/Zip{ index = 10.0590 value = 2.5414 dict = 1.00 all = 2.7864 } Zip/UnZip{ index = 0.0994 value = 0.3935 dict = 1.00 all = 0.3589 } ---------------------------- total value len = 2253.124866 GB avg = 0.199 KB (by entry num) total key len = 307.288972 GB avg = 0.027 KB total ukey len = 237.644734 GB avg = 0.015 KB total ukey num = 15.753161518 Billion total entry num = 11.294475466 Billion write speed all = 37.649698553 MB/sec (with seq num) write speed all = 36.321058346 MB/sec (without seq num) 2024/11/24-03:48:38.447347 139957315876416 [compaction_job.cc:1948] [default] [JOB 41032] Generated table #290865: 19764 keys, 1035589 bytes, temperature: kWarm 2024/11/24-03:48:38.447394 139957315876416 EVENT_LOG_v1 {"time_micros": 1732391318447367, "cf_name": "default", "job": 41032, "event": "table_file_creation", "file_number": 290865, "file_size": 1035589, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4306725368, "largest_seqno": 4310028747, "table_properties": {"data_size": 917648, "index_size": 39296, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 553392, "raw_average_key_size": 28, "raw_value_size": 2332152, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 19764, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391318, "oldest_key_time": 0, "file_creation_time": 1732391318, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290865, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:48:38.455580 139957315876416 [compaction_job.cc:703] job-41032: subcompact[0], size: 0.573041 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41032/att-00 [290849,290850,290851,290852,290853,290854,290855,290856,290857,290858,290859,290860,290861,290862,290863,290864,290865] 2024/11/24-03:48:38.554164 139957315876416 (Original Log Time 2024/11/24-03:47:39.923663) EVENT_LOG_v1 {"time_micros": 1732391259923636, "job": 41032, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [290846], "files_L3": [290761, 290762, 290763, 290764, 290765, 290778, 290779, 290780, 290781, 290782, 290783, 290784, 290785, 290786, 290787, 290788], "score": 0.620905, "input_data_size": 596747065, "oldest_snapshot_seqno": 4424771278} 2024/11/24-03:48:38.554168 139957315876416 (Original Log Time 2024/11/24-03:48:38.552750) [compaction_job.cc:2022] [default] [JOB 41032] Compacted 1@2 + 16@3 files to L3 => 573041058 bytes 2024/11/24-03:48:38.554170 139957315876416 (Original Log Time 2024/11/24-03:48:38.554091) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 17 0 0 0] max score 0.02, MB/sec: 10.2 rd, 9.8 wr, level 3, files in(1, 16) out(17 +0 blob) MB in(27.1, 542.0 +0.0 blob) out(546.5 +0.0 blob), read-write-amplify(41.2) write-amplify(20.2) OK, records in: 14614146, records dropped: 113791 output_compression: Snappy 2024/11/24-03:48:38.554175 139957315876416 (Original Log Time 2024/11/24-03:48:38.554135) EVENT_LOG_v1 {"time_micros": 1732391318554116, "cf": "default", "job": 41032, "event": "compaction_finished", "compaction_time_micros": 58531922, "compaction_time_cpu_micros": 23243707, "output_level": 3, "num_output_files": 17, "total_output_size": 573041058, "num_input_records": 14614146, "num_output_records": 14500355, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 435233519, "file_range_sync_nanos": 0, "file_fsync_nanos": 12027, "file_prepare_write_nanos": 41388, "lsm_state": [0, 0, 0, 17, 0, 0, 0]} 2024/11/24-03:48:38.675081 139957315876416 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732391259,"executesMs":58758,"compactionJobId":41032,"attempt":0,"compactionInputRawBytes":2119958688,"compactionInputZipBytes":596747065,"compactionOutputRawBytes":0,"compactionOutputZipBytes":573041058,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-8928adfc17e208adec949a1dac6fddae-10e449ac2ba5d7a0-00"}}, response = 2024/11/24-03:48:38.675164 139957315876416 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41032/att-00: olev 3, work 58.758 s, result 5.927 ms, install 1.443 ms, input{raw 1.974 GiB zip 569.102 MiB}