2024/11/22-17:33:11.219109 140024978429504 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00: fork to child time = 0.010092 sec 2024/11/22-17:33:11.219979 140024978429504 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00 2024/11/22-17:33:11.227380 140024978429504 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00 2024/11/22-17:33:11.228328 140024978429504 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-17:33:11.228805 140024978429504 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-17:33:11.275713 140024978429504 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-06171/att-00/MANIFEST-206347 2024/11/22-17:33:11.312016 140024978429504 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-06171/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 217822, last_sequence is 3426532593, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 217688 2024/11/22-17:33:11.312024 140024978429504 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-17:33:11.312157 140024978429504 [version_set.cc:5721] Creating manifest 217822 2024/11/22-17:33:11.344804 140024978429504 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-17:33:11.364134 140024978429504 [compaction_job.cc:2437] [default] [JOB 6171] Compacting 1@2 + 15@3 files to L3, score 0.92, subcompactions 1 : 1 2024/11/22-17:33:11.364157 140024978429504 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 4742 Base level 2, inputs: [217813(27MB)], [217763(35MB) 217764(35MB) 217765(36MB) 217766(36MB) 217767(36MB) 217768(35MB) 217769(35MB) 217770(35MB) 217771(35MB) 217772(37MB) 217773(35MB) 217774(36MB) 217775(37MB) 217776(36MB) 217782(36MB)] 2024/11/22-17:33:11.371880 140024978429504 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-17:33:11.371912 140024978429504 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-17:33:11.371933 140024978429504 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-17:33:11.371938 140024978429504 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-17:33:11.371949 140024978429504 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-17:33:11.901407 140024978429504 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217823.sst) /dev/shm/ToplingTemp/Topling-221496-jmPQWI 2024/11/22-17:33:12.502983 140024978429504 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217823.sst): first pass time = 0.60's, 223.129'MB/sec 2024/11/22-17:33:15.993343 140024978429504 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217823.sst): old prealloc_size = 147639500, real_size = 32925248 2024/11/22-17:33:15.993975 140024978429504 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217823.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921806 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0309 GB avg-val = 117.60 avg-zval = 33.55 2024/11/22-17:33:15.994821 140024978429504 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217823.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-17:33:17.355079 140024978429504 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217823.sst): second pass time = 3.49's, 31.067'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.36's, 27.179'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2883.124'MB/sec (index lex order gen) rebuild zvType time = 1.08's, 0.213'MB/sec write SST data time = 0.28's, 131.883'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 33022.182'MB/sec zip my value time = 3.49's, unzip length = 0.108'GB zip my value throughput = 31.067'MB/sec zip pipeline throughput = 91.505'MB/sec entries = 921806 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.60 avg-zval = 33.55 usrkeys = 921806 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.60 avg-zval = 33.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921806 bytes = 3226336 } seq expand size = 6452642 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.4443 value = 3.5050 dict = 1.00 all = 3.6265 } Zip/UnZip{ index = 0.0957 value = 0.2853 dict = 1.00 all = 0.2758 } ---------------------------- total value len = 0.114860 GB avg = 0.125 KB (by entry num) total key len = 0.025811 GB avg = 0.028 KB total ukey len = 0.018436 GB avg = 0.020 KB total ukey num = 0.000921806 Billion total entry num = 0.000921806 Billion write speed all = 0.004500840 MB/sec (with seq num) write speed all = 0.004264889 MB/sec (without seq num) 2024/11/22-17:33:17.391569 140024978429504 [compaction_job.cc:1948] [default] [JOB 6171] Generated table #217823: 921806 keys, 37010780 bytes, temperature: kWarm 2024/11/22-17:33:17.391683 140024978429504 EVENT_LOG_v1 {"time_micros": 1732267997391613, "cf_name": "default", "job": 6171, "event": "table_file_creation", "file_number": 217823, "file_size": 37010780, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702550, "largest_seqno": 3426493449, "table_properties": {"data_size": 30929456, "index_size": 1765184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25810568, "raw_average_key_size": 28, "raw_value_size": 108407196, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921806, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732267991, "oldest_key_time": 0, "file_creation_time": 1732267991, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 217823, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-17:33:17.447305 140024978429504 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217824.sst) /dev/shm/ToplingTemp/Topling-221496-DHiYrG 2024/11/22-17:33:19.130600 140024978429504 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217824.sst): first pass time = 1.68's, 79.737'MB/sec 2024/11/22-17:33:20.966290 140024978429504 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217824.sst): old prealloc_size = 147639500, real_size = 33393952 2024/11/22-17:33:20.966985 140024978429504 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217824.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921864 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0314 GB avg-val = 117.59 avg-zval = 34.07 2024/11/22-17:33:20.968019 140024978429504 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217824.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-17:33:22.277579 140024978429504 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217824.sst): second pass time = 1.83's, 59.083'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.31's, 28.582'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2599.834'MB/sec (index lex order gen) rebuild zvType time = 1.15's, 0.200'MB/sec write SST data time = 0.16's, 235.929'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 36942.853'MB/sec zip my value time = 1.83's, unzip length = 0.108'GB zip my value throughput = 59.083'MB/sec zip pipeline throughput = 59.083'MB/sec entries = 921864 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.59 avg-zval = 34.07 usrkeys = 921864 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.59 avg-zval = 34.07 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921864 bytes = 3226544 } seq expand size = 6453048 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5124 value = 3.4514 dict = 1.00 all = 3.5811 } Zip/UnZip{ index = 0.0951 value = 0.2897 dict = 1.00 all = 0.2792 } ---------------------------- total value len = 0.229718 GB avg = 0.125 KB (by entry num) total key len = 0.051623 GB avg = 0.028 KB total ukey len = 0.036873 GB avg = 0.020 KB total ukey num = 0.001843670 Billion total entry num = 0.001843670 Billion write speed all = 0.009000274 MB/sec (with seq num) write speed all = 0.008528433 MB/sec (without seq num) 2024/11/22-17:33:22.314990 140024978429504 [compaction_job.cc:1948] [default] [JOB 6171] Generated table #217824: 921864 keys, 37479252 bytes, temperature: kWarm 2024/11/22-17:33:22.315048 140024978429504 EVENT_LOG_v1 {"time_micros": 1732268002315017, "cf_name": "default", "job": 6171, "event": "table_file_creation", "file_number": 217824, "file_size": 37479252, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702561, "largest_seqno": 3426493460, "table_properties": {"data_size": 31409424, "index_size": 1753856, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25812192, "raw_average_key_size": 28, "raw_value_size": 108405544, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921864, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732267997, "oldest_key_time": 0, "file_creation_time": 1732267997, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 217824, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-17:33:22.351903 140024978429504 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217825.sst) /dev/shm/ToplingTemp/Topling-221496-DuMptI 2024/11/22-17:33:23.219878 140024978429504 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217825.sst): first pass time = 0.87's, 154.641'MB/sec 2024/11/22-17:33:24.604457 140024978429504 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217825.sst): old prealloc_size = 147639500, real_size = 33922720 2024/11/22-17:33:24.605027 140024978429504 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217825.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921942 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0319 GB avg-val = 117.58 avg-zval = 34.64 2024/11/22-17:33:24.605825 140024978429504 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217825.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-17:33:25.634876 140024978429504 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217825.sst): second pass time = 1.38's, 78.371'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 36.887'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3142.303'MB/sec (index lex order gen) rebuild zvType time = 0.90's, 0.256'MB/sec write SST data time = 0.13's, 294.797'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 36236.955'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.371'MB/sec zip pipeline throughput = 78.371'MB/sec entries = 921942 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.58 avg-zval = 34.64 usrkeys = 921942 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.58 avg-zval = 34.64 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921942 bytes = 3226816 } seq expand size = 6453594 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.5148 value = 3.3941 dict = 1.00 all = 3.5313 } Zip/UnZip{ index = 0.0951 value = 0.2946 dict = 1.00 all = 0.2832 } ---------------------------- total value len = 0.344575 GB avg = 0.125 KB (by entry num) total key len = 0.077437 GB avg = 0.028 KB total ukey len = 0.055312 GB avg = 0.020 KB total ukey num = 0.002765612 Billion total entry num = 0.002765612 Billion write speed all = 0.013498988 MB/sec (with seq num) write speed all = 0.012791275 MB/sec (without seq num) 2024/11/22-17:33:25.675169 140024978429504 [compaction_job.cc:1948] [default] [JOB 6171] Generated table #217825: 921942 keys, 38008220 bytes, temperature: kWarm 2024/11/22-17:33:25.675228 140024978429504 EVENT_LOG_v1 {"time_micros": 1732268005675198, "cf_name": "default", "job": 6171, "event": "table_file_creation", "file_number": 217825, "file_size": 38008220, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702575, "largest_seqno": 3426493477, "table_properties": {"data_size": 31938448, "index_size": 1753600, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25814376, "raw_average_key_size": 28, "raw_value_size": 108403464, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921942, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732268002, "oldest_key_time": 0, "file_creation_time": 1732268002, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 217825, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-17:33:25.714321 140024978429504 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217826.sst) /dev/shm/ToplingTemp/Topling-221496-JNGyXI 2024/11/22-17:33:26.643247 140024978429504 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217826.sst): first pass time = 0.93's, 144.493'MB/sec 2024/11/22-17:33:27.999087 140024978429504 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217826.sst): old prealloc_size = 147639500, real_size = 33949728 2024/11/22-17:33:27.999616 140024978429504 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217826.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921786 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0320 GB avg-val = 117.61 avg-zval = 34.68 2024/11/22-17:33:28.000370 140024978429504 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217826.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-17:33:29.037542 140024978429504 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217826.sst): second pass time = 1.36's, 79.995'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.04's, 36.627'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3457.563'MB/sec (index lex order gen) rebuild zvType time = 0.91's, 0.254'MB/sec write SST data time = 0.13's, 290.880'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 37229.966'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.995'MB/sec zip pipeline throughput = 79.995'MB/sec entries = 921786 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.61 avg-zval = 34.68 usrkeys = 921786 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.61 avg-zval = 34.68 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921786 bytes = 3226272 } seq expand size = 6452502 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.5131 value = 3.3914 dict = 1.00 all = 3.5288 } Zip/UnZip{ index = 0.0951 value = 0.2949 dict = 1.00 all = 0.2834 } ---------------------------- total value len = 0.459436 GB avg = 0.125 KB (by entry num) total key len = 0.103247 GB avg = 0.028 KB total ukey len = 0.073748 GB avg = 0.020 KB total ukey num = 0.003687398 Billion total entry num = 0.003687398 Billion write speed all = 0.017996672 MB/sec (with seq num) write speed all = 0.017053179 MB/sec (without seq num) 2024/11/22-17:33:29.070670 140024978429504 [compaction_job.cc:1948] [default] [JOB 6171] Generated table #217826: 921786 keys, 38035252 bytes, temperature: kWarm 2024/11/22-17:33:29.070734 140024978429504 EVENT_LOG_v1 {"time_micros": 1732268009070701, "cf_name": "default", "job": 6171, "event": "table_file_creation", "file_number": 217826, "file_size": 38035252, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702585, "largest_seqno": 3426494017, "table_properties": {"data_size": 31965520, "index_size": 1753600, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25810008, "raw_average_key_size": 28, "raw_value_size": 108407756, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921786, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732268005, "oldest_key_time": 0, "file_creation_time": 1732268005, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 217826, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-17:33:29.108839 140024978429504 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217827.sst) /dev/shm/ToplingTemp/Topling-221496-dned2G 2024/11/22-17:33:30.269423 140024978429504 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217827.sst): first pass time = 1.16's, 115.650'MB/sec 2024/11/22-17:33:31.839850 140024978429504 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217827.sst): old prealloc_size = 147639500, real_size = 34005904 2024/11/22-17:33:31.840481 140024978429504 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217827.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921823 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0320 GB avg-val = 117.60 avg-zval = 34.72 2024/11/22-17:33:31.841281 140024978429504 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217827.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-17:33:32.896465 140024978429504 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217827.sst): second pass time = 1.57's, 69.098'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.06's, 36.050'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2899.872'MB/sec (index lex order gen) rebuild zvType time = 0.92's, 0.250'MB/sec write SST data time = 0.13's, 283.200'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 33412.730'MB/sec zip my value time = 1.57's, unzip length = 0.108'GB zip my value throughput = 69.098'MB/sec zip pipeline throughput = 69.098'MB/sec entries = 921823 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.60 avg-zval = 34.72 usrkeys = 921823 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.60 avg-zval = 34.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921823 bytes = 3226400 } seq expand size = 6452761 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.4445 value = 3.3866 dict = 1.00 all = 3.5236 } Zip/UnZip{ index = 0.0957 value = 0.2953 dict = 1.00 all = 0.2838 } ---------------------------- total value len = 0.574295 GB avg = 0.125 KB (by entry num) total key len = 0.129058 GB avg = 0.028 KB total ukey len = 0.092184 GB avg = 0.020 KB total ukey num = 0.004609221 Billion total entry num = 0.004609221 Billion write speed all = 0.022493059 MB/sec (with seq num) write speed all = 0.021313845 MB/sec (without seq num) 2024/11/22-17:33:32.927758 140024978429504 [compaction_job.cc:1948] [default] [JOB 6171] Generated table #217827: 921823 keys, 38091140 bytes, temperature: kWarm 2024/11/22-17:33:32.927833 140024978429504 EVENT_LOG_v1 {"time_micros": 1732268012927796, "cf_name": "default", "job": 6171, "event": "table_file_creation", "file_number": 217827, "file_size": 38091140, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702604, "largest_seqno": 3426493937, "table_properties": {"data_size": 32010112, "index_size": 1765184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25811044, "raw_average_key_size": 28, "raw_value_size": 108406762, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921823, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732268009, "oldest_key_time": 0, "file_creation_time": 1732268009, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 217827, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-17:33:32.968175 140024978429504 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217828.sst) /dev/shm/ToplingTemp/Topling-221496-0g0yUG 2024/11/22-17:33:33.987930 140024978429504 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217828.sst): first pass time = 1.02's, 131.623'MB/sec 2024/11/22-17:33:35.343170 140024978429504 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217828.sst): old prealloc_size = 147639500, real_size = 33700608 2024/11/22-17:33:35.343732 140024978429504 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217828.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921772 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0317 GB avg-val = 117.61 avg-zval = 34.40 2024/11/22-17:33:35.344536 140024978429504 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217828.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-17:33:36.274547 140024978429504 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217828.sst): second pass time = 1.35's, 80.056'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.571'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3276.824'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.288'MB/sec write SST data time = 0.13's, 291.161'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 35333.964'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.056'MB/sec zip pipeline throughput = 80.056'MB/sec entries = 921772 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.61 avg-zval = 34.40 usrkeys = 921772 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.61 avg-zval = 34.40 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921772 bytes = 3226224 } seq expand size = 6452404 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.4443 value = 3.4193 dict = 1.00 all = 3.5520 } Zip/UnZip{ index = 0.0957 value = 0.2925 dict = 1.00 all = 0.2815 } ---------------------------- total value len = 0.689156 GB avg = 0.125 KB (by entry num) total key len = 0.154868 GB avg = 0.028 KB total ukey len = 0.110620 GB avg = 0.020 KB total ukey num = 0.005530993 Billion total entry num = 0.005530993 Billion write speed all = 0.026988737 MB/sec (with seq num) write speed all = 0.025573853 MB/sec (without seq num) 2024/11/22-17:33:36.309719 140024978429504 [compaction_job.cc:1948] [default] [JOB 6171] Generated table #217828: 921772 keys, 37786080 bytes, temperature: kWarm 2024/11/22-17:33:36.309792 140024978429504 EVENT_LOG_v1 {"time_micros": 1732268016309752, "cf_name": "default", "job": 6171, "event": "table_file_creation", "file_number": 217828, "file_size": 37786080, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702617, "largest_seqno": 3426493516, "table_properties": {"data_size": 31704880, "index_size": 1765120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25809616, "raw_average_key_size": 28, "raw_value_size": 108408124, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921772, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732268012, "oldest_key_time": 0, "file_creation_time": 1732268012, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 217828, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-17:33:36.380129 140024978429504 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217829.sst) /dev/shm/ToplingTemp/Topling-221496-kQRPYI 2024/11/22-17:33:37.433732 140024978429504 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217829.sst): first pass time = 1.05's, 127.394'MB/sec 2024/11/22-17:33:38.838367 140024978429504 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217829.sst): old prealloc_size = 147639500, real_size = 33229664 2024/11/22-17:33:38.839128 140024978429504 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217829.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921772 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0312 GB avg-val = 117.61 avg-zval = 33.90 2024/11/22-17:33:38.840048 140024978429504 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217829.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-17:33:39.884478 140024978429504 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217829.sst): second pass time = 1.40's, 77.224'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.05's, 35.670'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2372.689'MB/sec (index lex order gen) rebuild zvType time = 0.89's, 0.260'MB/sec write SST data time = 0.16's, 233.609'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33060.507'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.224'MB/sec zip pipeline throughput = 77.224'MB/sec entries = 921772 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.61 avg-zval = 33.90 usrkeys = 921772 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.61 avg-zval = 33.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921772 bytes = 3226224 } seq expand size = 6452404 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5110 value = 3.4696 dict = 1.00 all = 3.5969 } Zip/UnZip{ index = 0.0951 value = 0.2882 dict = 1.00 all = 0.2780 } ---------------------------- total value len = 0.804016 GB avg = 0.125 KB (by entry num) total key len = 0.180677 GB avg = 0.028 KB total ukey len = 0.129055 GB avg = 0.020 KB total ukey num = 0.006452765 Billion total entry num = 0.006452765 Billion write speed all = 0.031483213 MB/sec (with seq num) write speed all = 0.029832720 MB/sec (without seq num) 2024/11/22-17:33:39.917718 140024978429504 [compaction_job.cc:1948] [default] [JOB 6171] Generated table #217829: 921772 keys, 37314740 bytes, temperature: kWarm 2024/11/22-17:33:39.917792 140024978429504 EVENT_LOG_v1 {"time_micros": 1732268019917751, "cf_name": "default", "job": 6171, "event": "table_file_creation", "file_number": 217829, "file_size": 37314740, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702634, "largest_seqno": 3426493528, "table_properties": {"data_size": 31245136, "index_size": 1753920, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25809616, "raw_average_key_size": 28, "raw_value_size": 108408160, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921772, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732268016, "oldest_key_time": 0, "file_creation_time": 1732268016, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 217829, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-17:33:39.954746 140024978429504 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217830.sst) /dev/shm/ToplingTemp/Topling-221496-hVW4uE 2024/11/22-17:33:41.001671 140024978429504 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217830.sst): first pass time = 1.05's, 128.207'MB/sec 2024/11/22-17:33:42.637488 140024978429504 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217830.sst): old prealloc_size = 147639500, real_size = 32471168 2024/11/22-17:33:42.638064 140024978429504 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217830.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923909 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0305 GB avg-val = 117.27 avg-zval = 32.99 2024/11/22-17:33:42.638849 140024978429504 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217830.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-17:33:43.635277 140024978429504 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217830.sst): second pass time = 1.64's, 66.261'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.00's, 36.644'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3138.924'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.300'MB/sec write SST data time = 0.23's, 161.260'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 30977.986'MB/sec zip my value time = 1.64's, unzip length = 0.108'GB zip my value throughput = 66.261'MB/sec zip pipeline throughput = 97.432'MB/sec entries = 923909 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.27 avg-zval = 32.99 usrkeys = 923909 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.27 avg-zval = 32.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923909 bytes = 3233696 } seq expand size = 6467363 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5123 value = 3.5545 dict = 1.00 all = 3.6710 } Zip/UnZip{ index = 0.0951 value = 0.2813 dict = 1.00 all = 0.2724 } ---------------------------- total value len = 0.918832 GB avg = 0.125 KB (by entry num) total key len = 0.206547 GB avg = 0.028 KB total ukey len = 0.147533 GB avg = 0.020 KB total ukey num = 0.007376674 Billion total entry num = 0.007376674 Billion write speed all = 0.035976965 MB/sec (with seq num) write speed all = 0.034090380 MB/sec (without seq num) 2024/11/22-17:33:43.670789 140024978429504 [compaction_job.cc:1948] [default] [JOB 6171] Generated table #217830: 923909 keys, 36561850 bytes, temperature: kWarm 2024/11/22-17:33:43.670882 140024978429504 EVENT_LOG_v1 {"time_micros": 1732268023670837, "cf_name": "default", "job": 6171, "event": "table_file_creation", "file_number": 217830, "file_size": 36561850, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702649, "largest_seqno": 3426493555, "table_properties": {"data_size": 30482224, "index_size": 1757760, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25869452, "raw_average_key_size": 28, "raw_value_size": 108348278, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923909, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732268019, "oldest_key_time": 0, "file_creation_time": 1732268019, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 217830, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-17:33:43.735084 140024978429504 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217831.sst) /dev/shm/ToplingTemp/Topling-221496-6qAqPE 2024/11/22-17:33:44.634122 140024978429504 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217831.sst): first pass time = 0.90's, 149.300'MB/sec 2024/11/22-17:33:45.976284 140024978429504 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217831.sst): old prealloc_size = 147639500, real_size = 32721856 2024/11/22-17:33:45.976871 140024978429504 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217831.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922422 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0307 GB avg-val = 117.51 avg-zval = 33.32 2024/11/22-17:33:45.977601 140024978429504 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217831.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-17:33:46.858686 140024978429504 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217831.sst): second pass time = 1.34's, 80.823'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 41.719'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3106.577'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.306'MB/sec write SST data time = 0.13's, 286.147'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 21958.162'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.823'MB/sec zip pipeline throughput = 80.823'MB/sec entries = 922422 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.51 avg-zval = 33.32 usrkeys = 922422 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.51 avg-zval = 33.32 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922422 bytes = 3228496 } seq expand size = 6456954 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5138 value = 3.5264 dict = 1.00 all = 3.6460 } Zip/UnZip{ index = 0.0951 value = 0.2836 dict = 1.00 all = 0.2743 } ---------------------------- total value len = 1.033679 GB avg = 0.125 KB (by entry num) total key len = 0.232375 GB avg = 0.028 KB total ukey len = 0.165982 GB avg = 0.020 KB total ukey num = 0.008299096 Billion total entry num = 0.008299096 Billion write speed all = 0.040469991 MB/sec (with seq num) write speed all = 0.038347715 MB/sec (without seq num) 2024/11/22-17:33:46.893356 140024978429504 [compaction_job.cc:1948] [default] [JOB 6171] Generated table #217831: 922422 keys, 36812302 bytes, temperature: kWarm 2024/11/22-17:33:46.893422 140024978429504 EVENT_LOG_v1 {"time_micros": 1732268026893388, "cf_name": "default", "job": 6171, "event": "table_file_creation", "file_number": 217831, "file_size": 36812302, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868609, "largest_seqno": 3426493569, "table_properties": {"data_size": 30736368, "index_size": 1754688, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25827816, "raw_average_key_size": 28, "raw_value_size": 108389952, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922422, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732268023, "oldest_key_time": 0, "file_creation_time": 1732268023, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 217831, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-17:33:46.931706 140024978429504 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217832.sst) /dev/shm/ToplingTemp/Topling-221496-NzsVvF 2024/11/22-17:33:47.840509 140024978429504 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217832.sst): first pass time = 0.91's, 147.694'MB/sec 2024/11/22-17:33:49.306888 140024978429504 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217832.sst): old prealloc_size = 147639500, real_size = 37162672 2024/11/22-17:33:49.307420 140024978429504 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217832.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925550 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0352 GB avg-val = 117.01 avg-zval = 38.00 2024/11/22-17:33:49.307861 140024978429504 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217832.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-17:33:50.010508 140024978429504 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217832.sst): second pass time = 1.47's, 73.912'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.70's, 55.841'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3451.735'MB/sec (index lex order gen) rebuild zvType time = 0.57's, 0.403'MB/sec write SST data time = 0.13's, 305.288'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 30000.944'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.912'MB/sec zip pipeline throughput = 73.912'MB/sec entries = 925550 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.01 avg-zval = 38.00 usrkeys = 925550 avg-key = 20.00 avg-zkey = 1.90 avg-val = 119.63 avg-zval = 38.00 TagRS{ kinds = RS_Key0_TagN bytes = 144720 } TagArray{ size = 345876 bytes = 1124112 } seq expand size = 2421132 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0352 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.5130 value = 3.0794 dict = 1.00 all = 3.4161 } Zip/UnZip{ index = 0.0951 value = 0.3247 dict = 1.00 all = 0.2927 } ---------------------------- total value len = 1.144402 GB avg = 0.124 KB (by entry num) total key len = 0.258290 GB avg = 0.028 KB total ukey len = 0.184493 GB avg = 0.020 KB total ukey num = 0.009224646 Billion total entry num = 0.009224646 Billion write speed all = 0.044833199 MB/sec (with seq num) write speed all = 0.042474476 MB/sec (without seq num) 2024/11/22-17:33:50.052489 140024978429504 [compaction_job.cc:1948] [default] [JOB 6171] Generated table #217832: 925550 keys, 39290309 bytes, temperature: kWarm 2024/11/22-17:33:50.052552 140024978429504 EVENT_LOG_v1 {"time_micros": 1732268030052520, "cf_name": "default", "job": 6171, "event": "table_file_creation", "file_number": 217832, "file_size": 39290309, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700833, "table_properties": {"data_size": 35170336, "index_size": 1760768, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25915400, "raw_average_key_size": 28, "raw_value_size": 108302336, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925550, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732268026, "oldest_key_time": 0, "file_creation_time": 1732268026, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 217832, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-17:33:50.087044 140024978429504 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217833.sst) /dev/shm/ToplingTemp/Topling-221496-LPrPEI 2024/11/22-17:33:50.818059 140024978429504 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217833.sst): first pass time = 0.73's, 183.615'MB/sec 2024/11/22-17:33:52.380062 140024978429504 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217833.sst): old prealloc_size = 147639500, real_size = 36471728 2024/11/22-17:33:52.380592 140024978429504 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217833.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927087 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.77 avg-zval = 37.20 2024/11/22-17:33:52.380638 140024978429504 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217833.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-17:33:53.022833 140024978429504 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217833.sst): second pass time = 1.56's, 69.339'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 58.079'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3461.086'MB/sec (index lex order gen) rebuild zvType time = 0.52's, 0.450'MB/sec write SST data time = 0.13's, 294.620'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 30197.703'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.339'MB/sec zip pipeline throughput = 69.339'MB/sec entries = 927087 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.77 avg-zval = 37.20 usrkeys = 927087 avg-key = 20.00 avg-zkey = 1.89 avg-val = 116.77 avg-zval = 37.20 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5820 value = 3.1391 dict = 1.00 all = 3.5954 } Zip/UnZip{ index = 0.0945 value = 0.3186 dict = 1.00 all = 0.2781 } ---------------------------- total value len = 1.252662 GB avg = 0.123 KB (by entry num) total key len = 0.284249 GB avg = 0.028 KB total ukey len = 0.203035 GB avg = 0.020 KB total ukey num = 0.010151733 Billion total entry num = 0.010151733 Billion write speed all = 0.049118373 MB/sec (with seq num) write speed all = 0.046522845 MB/sec (without seq num) 2024/11/22-17:33:53.054515 140024978429504 [compaction_job.cc:1948] [default] [JOB 6171] Generated table #217833: 927087 keys, 37329978 bytes, temperature: kWarm 2024/11/22-17:33:53.054582 140024978429504 EVENT_LOG_v1 {"time_micros": 1732268033054548, "cf_name": "default", "job": 6171, "event": "table_file_creation", "file_number": 217833, "file_size": 37329978, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34487584, "index_size": 1752192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25958436, "raw_average_key_size": 28, "raw_value_size": 108259386, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927087, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732268030, "oldest_key_time": 0, "file_creation_time": 1732268030, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 217833, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-17:33:53.089072 140024978429504 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217834.sst) /dev/shm/ToplingTemp/Topling-221496-XuuE8E 2024/11/22-17:33:53.842067 140024978429504 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217834.sst): first pass time = 0.75's, 178.254'MB/sec 2024/11/22-17:33:55.333319 140024978429504 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217834.sst): old prealloc_size = 147639500, real_size = 36685280 2024/11/22-17:33:55.333882 140024978429504 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217834.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924878 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0347 GB avg-val = 117.12 avg-zval = 37.50 2024/11/22-17:33:55.334181 140024978429504 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217834.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-17:33:56.044248 140024978429504 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217834.sst): second pass time = 1.49's, 72.697'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.71's, 54.176'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3282.590'MB/sec (index lex order gen) rebuild zvType time = 0.58's, 0.396'MB/sec write SST data time = 0.13's, 304.398'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33153.676'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.697'MB/sec zip pipeline throughput = 72.697'MB/sec entries = 924878 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.12 avg-zval = 37.50 usrkeys = 924878 avg-key = 20.00 avg-zkey = 1.92 avg-val = 119.04 avg-zval = 37.50 TagRS{ kinds = RS_Key0_TagN bytes = 144600 } TagArray{ size = 253868 bytes = 825088 } seq expand size = 1777076 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0347 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.4435 value = 3.1232 dict = 1.00 all = 3.4849 } Zip/UnZip{ index = 0.0958 value = 0.3202 dict = 1.00 all = 0.2870 } ---------------------------- total value len = 1.362760 GB avg = 0.123 KB (by entry num) total key len = 0.310145 GB avg = 0.028 KB total ukey len = 0.221532 GB avg = 0.020 KB total ukey num = 0.011076611 Billion total entry num = 0.011076611 Billion write speed all = 0.053459494 MB/sec (with seq num) write speed all = 0.050627773 MB/sec (without seq num) 2024/11/22-17:33:56.079755 140024978429504 [compaction_job.cc:1948] [default] [JOB 6171] Generated table #217834: 924878 keys, 38514569 bytes, temperature: kWarm 2024/11/22-17:33:56.079830 140024978429504 EVENT_LOG_v1 {"time_micros": 1732268036079797, "cf_name": "default", "job": 6171, "event": "table_file_creation", "file_number": 217834, "file_size": 38514569, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3325795090, "table_properties": {"data_size": 34682704, "index_size": 1771200, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25896584, "raw_average_key_size": 28, "raw_value_size": 108321228, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924878, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732268033, "oldest_key_time": 0, "file_creation_time": 1732268033, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 217834, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-17:33:56.136328 140024978429504 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217835.sst) /dev/shm/ToplingTemp/Topling-221496-jVQdUH 2024/11/22-17:33:57.045678 140024978429504 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217835.sst): first pass time = 0.91's, 147.605'MB/sec 2024/11/22-17:33:58.395482 140024978429504 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217835.sst): old prealloc_size = 147639500, real_size = 34572256 2024/11/22-17:33:58.396077 140024978429504 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217835.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924645 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0326 GB avg-val = 117.16 avg-zval = 35.24 2024/11/22-17:33:58.396849 140024978429504 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217835.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-17:33:59.306819 140024978429504 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217835.sst): second pass time = 1.35's, 80.294'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 42.301'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3071.817'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.295'MB/sec write SST data time = 0.13's, 302.739'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 30810.974'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.294'MB/sec zip pipeline throughput = 80.294'MB/sec entries = 924645 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.16 avg-zval = 35.24 usrkeys = 924645 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.16 avg-zval = 35.24 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924645 bytes = 3120688 } seq expand size = 6472515 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0326 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.5123 value = 3.3248 dict = 1.00 all = 3.4817 } Zip/UnZip{ index = 0.0951 value = 0.3008 dict = 1.00 all = 0.2872 } ---------------------------- total value len = 1.477560 GB avg = 0.123 KB (by entry num) total key len = 0.336035 GB avg = 0.028 KB total ukey len = 0.240025 GB avg = 0.020 KB total ukey num = 0.012001256 Billion total entry num = 0.012001256 Billion write speed all = 0.057949364 MB/sec (with seq num) write speed all = 0.054881579 MB/sec (without seq num) 2024/11/22-17:33:59.340648 140024978429504 [compaction_job.cc:1948] [default] [JOB 6171] Generated table #217835: 924645 keys, 38549032 bytes, temperature: kWarm 2024/11/22-17:33:59.340710 140024978429504 EVENT_LOG_v1 {"time_micros": 1732268039340678, "cf_name": "default", "job": 6171, "event": "table_file_creation", "file_number": 217835, "file_size": 38549032, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3303265604, "largest_seqno": 3426492905, "table_properties": {"data_size": 32581712, "index_size": 1759168, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25890060, "raw_average_key_size": 28, "raw_value_size": 108327774, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924645, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732268036, "oldest_key_time": 0, "file_creation_time": 1732268036, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 217835, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-17:33:59.376619 140024978429504 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217836.sst) /dev/shm/ToplingTemp/Topling-221496-LXI14H 2024/11/22-17:34:00.306947 140024978429504 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217836.sst): first pass time = 0.93's, 144.276'MB/sec 2024/11/22-17:34:01.628734 140024978429504 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217836.sst): old prealloc_size = 147639500, real_size = 33859568 2024/11/22-17:34:01.629307 140024978429504 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217836.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921868 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0319 GB avg-val = 117.59 avg-zval = 34.56 2024/11/22-17:34:01.630132 140024978429504 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217836.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-17:34:02.548493 140024978429504 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217836.sst): second pass time = 1.32's, 82.055'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 41.256'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3201.398'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.290'MB/sec write SST data time = 0.13's, 302.632'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 30902.733'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.055'MB/sec zip pipeline throughput = 82.055'MB/sec entries = 921868 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.59 avg-zval = 34.56 usrkeys = 921868 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.59 avg-zval = 34.56 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921868 bytes = 3226560 } seq expand size = 6453076 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.4435 value = 3.4022 dict = 1.00 all = 3.5372 } Zip/UnZip{ index = 0.0958 value = 0.2939 dict = 1.00 all = 0.2827 } ---------------------------- total value len = 1.592419 GB avg = 0.123 KB (by entry num) total key len = 0.361847 GB avg = 0.028 KB total ukey len = 0.258462 GB avg = 0.020 KB total ukey num = 0.012923124 Billion total entry num = 0.012923124 Billion write speed all = 0.062437719 MB/sec (with seq num) write speed all = 0.059134626 MB/sec (without seq num) 2024/11/22-17:34:02.586175 140024978429504 [compaction_job.cc:1948] [default] [JOB 6171] Generated table #217836: 921868 keys, 37944908 bytes, temperature: kWarm 2024/11/22-17:34:02.586242 140024978429504 EVENT_LOG_v1 {"time_micros": 1732268042586208, "cf_name": "default", "job": 6171, "event": "table_file_creation", "file_number": 217836, "file_size": 37944908, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701386, "largest_seqno": 3426492530, "table_properties": {"data_size": 31863456, "index_size": 1765440, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25812304, "raw_average_key_size": 28, "raw_value_size": 108405512, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921868, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732268039, "oldest_key_time": 0, "file_creation_time": 1732268039, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 217836, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-17:34:02.621250 140024978429504 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217837.sst) /dev/shm/ToplingTemp/Topling-221496-vPi0TE 2024/11/22-17:34:03.453038 140024978429504 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217837.sst): first pass time = 0.83's, 161.368'MB/sec 2024/11/22-17:34:04.813189 140024978429504 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217837.sst): old prealloc_size = 147639500, real_size = 35121056 2024/11/22-17:34:04.813770 140024978429504 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217837.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927178 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0331 GB avg-val = 116.76 avg-zval = 35.73 2024/11/22-17:34:04.814508 140024978429504 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217837.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-17:34:05.697013 140024978429504 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217837.sst): second pass time = 1.36's, 79.657'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 44.115'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3112.124'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.306'MB/sec write SST data time = 0.13's, 309.735'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32756.557'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.657'MB/sec zip pipeline throughput = 79.657'MB/sec entries = 927178 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.76 avg-zval = 35.73 usrkeys = 927178 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.76 avg-zval = 35.73 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927178 bytes = 3013344 } seq expand size = 6490246 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0331 GB dict = 1.08 MB all = 0.0390 GB } UnZip/Zip{ index = 10.5132 value = 3.2681 dict = 1.00 all = 3.4425 } Zip/UnZip{ index = 0.0951 value = 0.3060 dict = 1.00 all = 0.2905 } ---------------------------- total value len = 1.707166 GB avg = 0.123 KB (by entry num) total key len = 0.387808 GB avg = 0.028 KB total ukey len = 0.277006 GB avg = 0.020 KB total ukey num = 0.013850302 Billion total entry num = 0.013850302 Billion write speed all = 0.066926532 MB/sec (with seq num) write speed all = 0.063386813 MB/sec (without seq num) 2024/11/22-17:34:05.731319 140024978429504 [compaction_job.cc:1948] [default] [JOB 6171] Generated table #217837: 927178 keys, 38988776 bytes, temperature: kWarm 2024/11/22-17:34:05.731387 140024978429504 EVENT_LOG_v1 {"time_micros": 1732268045731353, "cf_name": "default", "job": 6171, "event": "table_file_creation", "file_number": 217837, "file_size": 38988776, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774735, "largest_seqno": 3426492563, "table_properties": {"data_size": 33125264, "index_size": 1763840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25960984, "raw_average_key_size": 28, "raw_value_size": 108256884, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927178, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732268042, "oldest_key_time": 0, "file_creation_time": 1732268042, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 217837, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-17:34:05.772336 140024978429504 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217838.sst) /dev/shm/ToplingTemp/Topling-221496-6ttYnG 2024/11/22-17:34:05.848676 140024978429504 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217838.sst): first pass time = 0.08's, 232.236'MB/sec 2024/11/22-17:34:06.009038 140024978429504 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217838.sst): old prealloc_size = 147639500, real_size = 5207568 2024/11/22-17:34:06.009161 140024978429504 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217838.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 122159 prefix = 4 raw-key = 0.0020 GB zip-key = 0.0002 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0143 GB zip-val = 0.0049 GB avg-val = 117.04 avg-zval = 40.45 2024/11/22-17:34:06.009292 140024978429504 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217838.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.018 GB 2024/11/22-17:34:06.127885 140024978429504 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00/217838.sst): second pass time = 0.16's, 89.489'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.12's, 48.052'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2403.473'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.302'MB/sec write SST data time = 0.02's, 326.949'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.150'MB, throughput = 4468.569'MB/sec zip my value time = 0.16's, unzip length = 0.014'GB zip my value throughput = 89.489'MB/sec zip pipeline throughput = 89.489'MB/sec entries = 122159 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.04 avg-zval = 40.45 usrkeys = 122159 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.04 avg-zval = 40.45 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 122159 bytes = 381760 } seq expand size = 855113 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0024 GB value = 0.0143 GB dict = 0.15 MB all = 0.0177 GB } __ZipSize{ index = 0.0002 GB value = 0.0049 GB dict = 0.15 MB all = 0.0057 GB } UnZip/Zip{ index = 10.3848 value = 2.8934 dict = 1.00 all = 3.1029 } Zip/UnZip{ index = 0.0963 value = 0.3456 dict = 1.00 all = 0.3223 } ---------------------------- total value len = 1.722319 GB avg = 0.123 KB (by entry num) total key len = 0.391229 GB avg = 0.028 KB total ukey len = 0.279449 GB avg = 0.020 KB total ukey num = 0.013972461 Billion total entry num = 0.013972461 Billion write speed all = 0.067518957 MB/sec (with seq num) write speed all = 0.063948067 MB/sec (without seq num) 2024/11/22-17:34:06.140839 140024978429504 [compaction_job.cc:1948] [default] [JOB 6171] Generated table #217838: 122159 keys, 5710199 bytes, temperature: kWarm 2024/11/22-17:34:06.140903 140024978429504 EVENT_LOG_v1 {"time_micros": 1732268046140867, "cf_name": "default", "job": 6171, "event": "table_file_creation", "file_number": 217838, "file_size": 5710199, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3406044294, "largest_seqno": 3424973329, "table_properties": {"data_size": 4941568, "index_size": 235264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3420452, "raw_average_key_size": 28, "raw_value_size": 14297958, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 122159, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732268045, "oldest_key_time": 0, "file_creation_time": 1732268045, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 217838, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-17:34:06.152038 140024978429504 [compaction_job.cc:703] job-06171: subcompact[0], size: 0.573427 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06171/att-00 [217823,217824,217825,217826,217827,217828,217829,217830,217831,217832,217833,217834,217835,217836,217837,217838] 2024/11/22-17:34:06.251830 140024978429504 (Original Log Time 2024/11/22-17:33:11.364218) EVENT_LOG_v1 {"time_micros": 1732267991364174, "job": 6171, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [217813], "files_L3": [217763, 217764, 217765, 217766, 217767, 217768, 217769, 217770, 217771, 217772, 217773, 217774, 217775, 217776, 217782], "score": 0.920464, "input_data_size": 595732494, "oldest_snapshot_seqno": 3426532588} 2024/11/22-17:34:06.251834 140024978429504 (Original Log Time 2024/11/22-17:34:06.250030) [compaction_job.cc:2022] [default] [JOB 6171] Compacted 1@2 + 15@3 files to L3 => 573427387 bytes 2024/11/22-17:34:06.251835 140024978429504 (Original Log Time 2024/11/22-17:34:06.251753) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 16 0 0 0] max score 0.02, MB/sec: 10.9 rd, 10.5 wr, level 3, files in(1, 15) out(16 +0 blob) MB in(27.1, 541.0 +0.0 blob) out(546.9 +0.0 blob), read-write-amplify(41.1) write-amplify(20.2) OK, records in: 14083902, records dropped: 111441 output_compression: Snappy 2024/11/22-17:34:06.251841 140024978429504 (Original Log Time 2024/11/22-17:34:06.251808) EVENT_LOG_v1 {"time_micros": 1732268046251784, "cf": "default", "job": 6171, "event": "compaction_finished", "compaction_time_micros": 54787828, "compaction_time_cpu_micros": 23872743, "output_level": 3, "num_output_files": 16, "total_output_size": 573427387, "num_input_records": 14083902, "num_output_records": 13972461, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 376821412, "file_range_sync_nanos": 0, "file_fsync_nanos": 14790, "file_prepare_write_nanos": 37986, "lsm_state": [0, 0, 0, 16, 0, 0, 0]} 2024/11/22-17:34:06.393179 140024978429504 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732267991,"executesMs":55025,"compactionJobId":6171,"attempt":0,"compactionInputRawBytes":2046809844,"compactionInputZipBytes":595732494,"compactionOutputRawBytes":0,"compactionOutputZipBytes":573427387,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-7d10210dc5337f5be086fe9e242fac95-15aba5a709e5f745-00"}}, response = 2024/11/22-17:34:06.393257 140024978429504 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-06171/att-00: olev 3, work 55.026 s, result 4.924 ms, install 1.832 ms, input{raw 1.906 GiB zip 568.135 MiB}