2024/11/22-22:27:56.949706 140025171461696 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00: fork to child time = 0.009196 sec 2024/11/22-22:27:56.950537 140025171461696 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00 2024/11/22-22:27:56.957958 140025171461696 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00 2024/11/22-22:27:56.959115 140025171461696 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-22:27:56.959422 140025171461696 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-22:27:57.006068 140025171461696 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-12764/att-00/MANIFEST-206347 2024/11/22-22:27:57.040200 140025171461696 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-12764/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 231321, last_sequence is 3609627419, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 231118 2024/11/22-22:27:57.040210 140025171461696 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-22:27:57.040315 140025171461696 [version_set.cc:5721] Creating manifest 231321 2024/11/22-22:27:57.071631 140025171461696 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-22:27:57.092272 140025171461696 [compaction_job.cc:2437] [default] [JOB 12764] Compacting 1@2 + 14@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/22-22:27:57.092295 140025171461696 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 9600 Base level 2, inputs: [231253(27MB)], [231318(36MB) 231319(6578KB) 231222(35MB) 231223(9194KB) 231184(36MB) 231185(34MB) 231186(35MB) 231187(35MB) 231188(35MB) 231189(36MB) 231190(34MB) 231198(35MB) 231199(36MB) 231283(35MB)] 2024/11/22-22:27:57.099938 140025171461696 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-22:27:57.099975 140025171461696 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-22:27:57.099995 140025171461696 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-22:27:57.100001 140025171461696 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-22:27:57.100013 140025171461696 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-22:27:57.564996 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231322.sst) /dev/shm/ToplingTemp/Topling-315261-vuevwL 2024/11/22-22:27:58.120927 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231322.sst): first pass time = 0.56's, 241.450'MB/sec 2024/11/22-22:28:00.170467 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231322.sst): old prealloc_size = 147639500, real_size = 34061856 2024/11/22-22:28:00.171060 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231322.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927074 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0321 GB avg-val = 116.78 avg-zval = 34.58 2024/11/22-22:28:00.171856 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231322.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:28:01.107447 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231322.sst): second pass time = 2.05's, 52.845'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 40.728'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3108.693'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.289'MB/sec write SST data time = 0.13's, 285.982'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 29844.730'MB/sec zip my value time = 2.05's, unzip length = 0.108'GB zip my value throughput = 52.845'MB/sec zip pipeline throughput = 77.309'MB/sec entries = 927074 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.78 avg-zval = 34.58 usrkeys = 927074 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.78 avg-zval = 34.58 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927074 bytes = 3244768 } seq expand size = 6489518 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0321 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.4427 value = 3.3774 dict = 1.00 all = 3.5171 } Zip/UnZip{ index = 0.0958 value = 0.2961 dict = 1.00 all = 0.2843 } ---------------------------- total value len = 0.114749 GB avg = 0.124 KB (by entry num) total key len = 0.025958 GB avg = 0.028 KB total ukey len = 0.018541 GB avg = 0.020 KB total ukey num = 0.000927074 Billion total entry num = 0.000927074 Billion write speed all = 0.002875214 MB/sec (with seq num) write speed all = 0.002723663 MB/sec (without seq num) 2024/11/22-22:28:01.143461 140025171461696 [compaction_job.cc:1948] [default] [JOB 12764] Generated table #231322: 927074 keys, 38161210 bytes, temperature: kWarm 2024/11/22-22:28:01.143556 140025171461696 EVENT_LOG_v1 {"time_micros": 1732285681143500, "cf_name": "default", "job": 12764, "event": "table_file_creation", "file_number": 231322, "file_size": 38161210, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3470221528, "largest_seqno": 3609091058, "table_properties": {"data_size": 32054352, "index_size": 1775552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25958072, "raw_average_key_size": 28, "raw_value_size": 108259668, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927074, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732285677, "oldest_key_time": 0, "file_creation_time": 1732285677, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 231322, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:28:01.183799 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231323.sst) /dev/shm/ToplingTemp/Topling-315261-o2W4cL 2024/11/22-22:28:02.504459 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231323.sst): first pass time = 1.32's, 101.632'MB/sec 2024/11/22-22:28:03.843103 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231323.sst): old prealloc_size = 147639500, real_size = 32510880 2024/11/22-22:28:03.843659 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231323.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924890 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0305 GB avg-val = 117.12 avg-zval = 33.00 2024/11/22-22:28:03.844433 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231323.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:28:04.748057 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231323.sst): second pass time = 1.34's, 80.967'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 40.449'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3294.940'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.298'MB/sec write SST data time = 0.13's, 286.426'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 26040.365'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.967'MB/sec zip pipeline throughput = 80.967'MB/sec entries = 924890 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.12 avg-zval = 33.00 usrkeys = 924890 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.12 avg-zval = 33.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924890 bytes = 3237136 } seq expand size = 6474230 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5158 value = 3.5491 dict = 1.00 all = 3.6668 } Zip/UnZip{ index = 0.0951 value = 0.2818 dict = 1.00 all = 0.2727 } ---------------------------- total value len = 0.229544 GB avg = 0.124 KB (by entry num) total key len = 0.051855 GB avg = 0.028 KB total ukey len = 0.037039 GB avg = 0.020 KB total ukey num = 0.001851964 Billion total entry num = 0.001851964 Billion write speed all = 0.005749690 MB/sec (with seq num) write speed all = 0.005446968 MB/sec (without seq num) 2024/11/22-22:28:04.782662 140025171461696 [compaction_job.cc:1948] [default] [JOB 12764] Generated table #231323: 924890 keys, 36603862 bytes, temperature: kWarm 2024/11/22-22:28:04.782727 140025171461696 EVENT_LOG_v1 {"time_micros": 1732285684782694, "cf_name": "default", "job": 12764, "event": "table_file_creation", "file_number": 231323, "file_size": 36603862, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3609090143, "table_properties": {"data_size": 30520464, "index_size": 1759040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25896920, "raw_average_key_size": 28, "raw_value_size": 108320920, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924890, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732285681, "oldest_key_time": 0, "file_creation_time": 1732285681, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 231323, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:28:04.818146 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231324.sst) /dev/shm/ToplingTemp/Topling-315261-oJn5nN 2024/11/22-22:28:05.760633 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231324.sst): first pass time = 0.94's, 142.415'MB/sec 2024/11/22-22:28:07.126938 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231324.sst): old prealloc_size = 147639500, real_size = 33911728 2024/11/22-22:28:07.127490 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231324.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931288 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0319 GB avg-val = 116.12 avg-zval = 34.26 2024/11/22-22:28:07.128250 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231324.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:28:08.043317 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231324.sst): second pass time = 1.37's, 79.217'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 41.366'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3341.860'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.295'MB/sec write SST data time = 0.13's, 297.595'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33233.235'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.217'MB/sec zip pipeline throughput = 79.217'MB/sec entries = 931288 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.12 avg-zval = 34.26 usrkeys = 931288 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.12 avg-zval = 34.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931288 bytes = 3143104 } seq expand size = 6519016 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.5140 value = 3.3893 dict = 1.00 all = 3.5408 } Zip/UnZip{ index = 0.0951 value = 0.2951 dict = 1.00 all = 0.2824 } ---------------------------- total value len = 0.344205 GB avg = 0.124 KB (by entry num) total key len = 0.077931 GB avg = 0.028 KB total ukey len = 0.055665 GB avg = 0.020 KB total ukey num = 0.002783252 Billion total entry num = 0.002783252 Billion write speed all = 0.008624714 MB/sec (with seq num) write speed all = 0.008169795 MB/sec (without seq num) 2024/11/22-22:28:08.077839 140025171461696 [compaction_job.cc:1948] [default] [JOB 12764] Generated table #231324: 931288 keys, 37906272 bytes, temperature: kWarm 2024/11/22-22:28:08.077904 140025171461696 EVENT_LOG_v1 {"time_micros": 1732285688077871, "cf_name": "default", "job": 12764, "event": "table_file_creation", "file_number": 231324, "file_size": 37906272, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3504428200, "largest_seqno": 3609090963, "table_properties": {"data_size": 31907232, "index_size": 1771520, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26076064, "raw_average_key_size": 28, "raw_value_size": 108141776, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931288, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732285684, "oldest_key_time": 0, "file_creation_time": 1732285684, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 231324, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:28:08.113634 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231325.sst) /dev/shm/ToplingTemp/Topling-315261-dExnpN 2024/11/22-22:28:08.953587 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231325.sst): first pass time = 0.84's, 159.800'MB/sec 2024/11/22-22:28:10.291113 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231325.sst): old prealloc_size = 147639500, real_size = 34917696 2024/11/22-22:28:10.291739 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231325.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1084932 prefix = 4 raw-key = 0.0174 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1038 GB zip-val = 0.0326 GB avg-val = 95.71 avg-zval = 30.02 2024/11/22-22:28:10.292106 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231325.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:28:11.006549 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231325.sst): second pass time = 1.34's, 77.680'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.72's, 51.529'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3430.810'MB/sec (index lex order gen) rebuild zvType time = 0.57's, 0.473'MB/sec write SST data time = 0.14's, 260.636'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.035'MB, throughput = 30835.135'MB/sec zip my value time = 1.34's, unzip length = 0.104'GB zip my value throughput = 77.680'MB/sec zip pipeline throughput = 77.680'MB/sec entries = 1084932 avg-key = 28.00 avg-zkey = 1.91 avg-val = 95.71 avg-zval = 30.02 usrkeys = 1084932 avg-key = 20.00 avg-zkey = 1.91 avg-val = 97.78 avg-zval = 30.02 TagRS{ kinds = RS_Key0_TagN bytes = 169640 } TagArray{ size = 320693 bytes = 1002176 } seq expand size = 2244851 multi value expand size = 0 cnt WriteAppend = 19 UnZipSize{ index = 0.0217 GB value = 0.1038 GB dict = 1.04 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0326 GB dict = 1.04 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4471 value = 3.1883 dict = 1.00 all = 3.6408 } Zip/UnZip{ index = 0.0957 value = 0.3136 dict = 1.00 all = 0.2747 } ---------------------------- total value len = 0.450290 GB avg = 0.116 KB (by entry num) total key len = 0.108309 GB avg = 0.028 KB total ukey len = 0.077364 GB avg = 0.020 KB total ukey num = 0.003868184 Billion total entry num = 0.003868184 Billion write speed all = 0.011412109 MB/sec (with seq num) write speed all = 0.010779897 MB/sec (without seq num) 2024/11/22-22:28:11.041938 140025171461696 [compaction_job.cc:1948] [default] [JOB 12764] Generated table #231325: 1084932 keys, 36864548 bytes, temperature: kWarm 2024/11/22-22:28:11.042003 140025171461696 EVENT_LOG_v1 {"time_micros": 1732285691041971, "cf_name": "default", "job": 12764, "event": "table_file_creation", "file_number": 231325, "file_size": 36864548, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3609090877, "table_properties": {"data_size": 32569264, "index_size": 2076992, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30378096, "raw_average_key_size": 28, "raw_value_size": 103839777, "raw_average_value_size": 95, "num_data_blocks": 1, "num_entries": 1084932, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732285688, "oldest_key_time": 0, "file_creation_time": 1732285688, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 231325, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:28:11.077442 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231326.sst) /dev/shm/ToplingTemp/Topling-315261-ojeg2K 2024/11/22-22:28:11.961954 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231326.sst): first pass time = 0.88's, 151.749'MB/sec 2024/11/22-22:28:13.418553 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231326.sst): old prealloc_size = 147639500, real_size = 36503152 2024/11/22-22:28:13.419105 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231326.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924410 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 117.19 avg-zval = 37.34 2024/11/22-22:28:13.419463 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231326.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:28:14.145422 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231326.sst): second pass time = 1.46's, 74.413'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.73's, 53.048'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3315.492'MB/sec (index lex order gen) rebuild zvType time = 0.60's, 0.385'MB/sec write SST data time = 0.13's, 306.065'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 35508.438'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 74.413'MB/sec zip pipeline throughput = 74.413'MB/sec entries = 924410 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.19 avg-zval = 37.34 usrkeys = 924410 avg-key = 20.00 avg-zkey = 1.90 avg-val = 119.47 avg-zval = 37.34 TagRS{ kinds = RS_Key0_TagN bytes = 144520 } TagArray{ size = 300246 bytes = 1050880 } seq expand size = 2101722 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.5138 value = 3.1389 dict = 1.00 all = 3.4809 } Zip/UnZip{ index = 0.0951 value = 0.3186 dict = 1.00 all = 0.2873 } ---------------------------- total value len = 0.560726 GB avg = 0.117 KB (by entry num) total key len = 0.134193 GB avg = 0.028 KB total ukey len = 0.095852 GB avg = 0.020 KB total ukey num = 0.004792594 Billion total entry num = 0.004792594 Billion write speed all = 0.014196192 MB/sec (with seq num) write speed all = 0.013412945 MB/sec (without seq num) 2024/11/22-22:28:14.179733 140025171461696 [compaction_job.cc:1948] [default] [JOB 12764] Generated table #231326: 924410 keys, 38558459 bytes, temperature: kWarm 2024/11/22-22:28:14.179807 140025171461696 EVENT_LOG_v1 {"time_micros": 1732285694179765, "cf_name": "default", "job": 12764, "event": "table_file_creation", "file_number": 231326, "file_size": 38558459, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3439129127, "table_properties": {"data_size": 34513376, "index_size": 1758464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25883480, "raw_average_key_size": 28, "raw_value_size": 108334380, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924410, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732285691, "oldest_key_time": 0, "file_creation_time": 1732285691, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 231326, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:28:14.216173 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231327.sst) /dev/shm/ToplingTemp/Topling-315261-FiNLkK 2024/11/22-22:28:15.119543 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231327.sst): first pass time = 0.90's, 148.582'MB/sec 2024/11/22-22:28:16.447162 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231327.sst): old prealloc_size = 147639500, real_size = 32013520 2024/11/22-22:28:16.447690 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231327.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924920 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0300 GB avg-val = 117.11 avg-zval = 32.46 2024/11/22-22:28:16.448463 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231327.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:28:17.320881 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231327.sst): second pass time = 1.33's, 81.653'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 41.326'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3473.569'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.310'MB/sec write SST data time = 0.13's, 283.601'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32594.212'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.653'MB/sec zip pipeline throughput = 81.653'MB/sec entries = 924920 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.11 avg-zval = 32.46 usrkeys = 924920 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.11 avg-zval = 32.46 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924920 bytes = 3237232 } seq expand size = 6474440 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.5135 value = 3.6079 dict = 1.00 all = 3.7173 } Zip/UnZip{ index = 0.0951 value = 0.2772 dict = 1.00 all = 0.2690 } ---------------------------- total value len = 0.675520 GB avg = 0.118 KB (by entry num) total key len = 0.160090 GB avg = 0.028 KB total ukey len = 0.114350 GB avg = 0.020 KB total ukey num = 0.005717514 Billion total entry num = 0.005717514 Billion write speed all = 0.017069225 MB/sec (with seq num) write speed all = 0.016134880 MB/sec (without seq num) 2024/11/22-22:28:17.353167 140025171461696 [compaction_job.cc:1948] [default] [JOB 12764] Generated table #231327: 924920 keys, 36106446 bytes, temperature: kWarm 2024/11/22-22:28:17.353229 140025171461696 EVENT_LOG_v1 {"time_micros": 1732285697353197, "cf_name": "default", "job": 12764, "event": "table_file_creation", "file_number": 231327, "file_size": 36106446, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3439129128, "largest_seqno": 3609091650, "table_properties": {"data_size": 30022592, "index_size": 1759488, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25897760, "raw_average_key_size": 28, "raw_value_size": 108319984, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924920, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732285694, "oldest_key_time": 0, "file_creation_time": 1732285694, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 231327, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:28:17.387732 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231328.sst) /dev/shm/ToplingTemp/Topling-315261-bEScEM 2024/11/22-22:28:18.207510 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231328.sst): first pass time = 0.82's, 163.734'MB/sec 2024/11/22-22:28:19.536786 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231328.sst): old prealloc_size = 147639500, real_size = 32787104 2024/11/22-22:28:19.537340 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231328.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924924 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0308 GB avg-val = 117.11 avg-zval = 33.28 2024/11/22-22:28:19.538111 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231328.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:28:20.438704 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231328.sst): second pass time = 1.33's, 81.560'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 40.891'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3343.184'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.299'MB/sec write SST data time = 0.13's, 288.637'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 30359.684'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.560'MB/sec zip pipeline throughput = 81.560'MB/sec entries = 924924 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.11 avg-zval = 33.28 usrkeys = 924924 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.11 avg-zval = 33.28 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924924 bytes = 3237248 } seq expand size = 6474468 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4448 value = 3.5186 dict = 1.00 all = 3.6393 } Zip/UnZip{ index = 0.0957 value = 0.2842 dict = 1.00 all = 0.2748 } ---------------------------- total value len = 0.790315 GB avg = 0.119 KB (by entry num) total key len = 0.185988 GB avg = 0.028 KB total ukey len = 0.132849 GB avg = 0.020 KB total ukey num = 0.006642438 Billion total entry num = 0.006642438 Billion write speed all = 0.019941911 MB/sec (with seq num) write speed all = 0.018856487 MB/sec (without seq num) 2024/11/22-22:28:20.470328 140025171461696 [compaction_job.cc:1948] [default] [JOB 12764] Generated table #231328: 924924 keys, 36880006 bytes, temperature: kWarm 2024/11/22-22:28:20.470391 140025171461696 EVENT_LOG_v1 {"time_micros": 1732285700470360, "cf_name": "default", "job": 12764, "event": "table_file_creation", "file_number": 231328, "file_size": 36880006, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3365258144, "largest_seqno": 3609091669, "table_properties": {"data_size": 30784592, "index_size": 1771072, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25897872, "raw_average_key_size": 28, "raw_value_size": 108319972, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924924, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732285697, "oldest_key_time": 0, "file_creation_time": 1732285697, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 231328, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:28:20.507345 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231329.sst) /dev/shm/ToplingTemp/Topling-315261-7brhkN 2024/11/22-22:28:21.313478 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231329.sst): first pass time = 0.81's, 166.504'MB/sec 2024/11/22-22:28:22.672707 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231329.sst): old prealloc_size = 147639500, real_size = 34552320 2024/11/22-22:28:22.673280 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231329.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926799 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0325 GB avg-val = 116.82 avg-zval = 35.12 2024/11/22-22:28:22.674453 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231329.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:28:23.587646 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231329.sst): second pass time = 1.36's, 79.694'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 42.371'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3218.842'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.295'MB/sec write SST data time = 0.13's, 297.940'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35060.293'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.694'MB/sec zip pipeline throughput = 79.694'MB/sec entries = 926799 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.82 avg-zval = 35.12 usrkeys = 926799 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.82 avg-zval = 35.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926799 bytes = 3359664 } seq expand size = 6487593 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0388 GB } UnZip/Zip{ index = 10.4460 value = 3.3266 dict = 1.00 all = 3.4622 } Zip/UnZip{ index = 0.0957 value = 0.3006 dict = 1.00 all = 0.2888 } ---------------------------- total value len = 0.905070 GB avg = 0.120 KB (by entry num) total key len = 0.211939 GB avg = 0.028 KB total ukey len = 0.151385 GB avg = 0.020 KB total ukey num = 0.007569237 Billion total entry num = 0.007569237 Billion write speed all = 0.022814483 MB/sec (with seq num) write speed all = 0.021577692 MB/sec (without seq num) 2024/11/22-22:28:23.623484 140025171461696 [compaction_job.cc:1948] [default] [JOB 12764] Generated table #231329: 926799 keys, 38766514 bytes, temperature: kWarm 2024/11/22-22:28:23.623548 140025171461696 EVENT_LOG_v1 {"time_micros": 1732285703623516, "cf_name": "default", "job": 12764, "event": "table_file_creation", "file_number": 231329, "file_size": 38766514, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3609091700, "table_properties": {"data_size": 32545968, "index_size": 1774464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25950372, "raw_average_key_size": 28, "raw_value_size": 108267362, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926799, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732285700, "oldest_key_time": 0, "file_creation_time": 1732285700, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 231329, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:28:23.662616 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231330.sst) /dev/shm/ToplingTemp/Topling-315261-rzlBgM 2024/11/22-22:28:24.545547 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231330.sst): first pass time = 0.88's, 152.021'MB/sec 2024/11/22-22:28:25.877758 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231330.sst): old prealloc_size = 147639500, real_size = 33623120 2024/11/22-22:28:25.878304 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231330.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925147 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 117.08 avg-zval = 34.19 2024/11/22-22:28:25.879683 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231330.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:28:26.823801 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231330.sst): second pass time = 1.33's, 81.346'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.869'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3423.117'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.283'MB/sec write SST data time = 0.13's, 293.496'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35976.171'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.346'MB/sec zip pipeline throughput = 81.346'MB/sec entries = 925147 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.08 avg-zval = 34.19 usrkeys = 925147 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.08 avg-zval = 34.19 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925147 bytes = 3238032 } seq expand size = 6476029 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5115 value = 3.4242 dict = 1.00 all = 3.5586 } Zip/UnZip{ index = 0.0951 value = 0.2920 dict = 1.00 all = 0.2810 } ---------------------------- total value len = 1.019859 GB avg = 0.120 KB (by entry num) total key len = 0.237843 GB avg = 0.028 KB total ukey len = 0.169888 GB avg = 0.020 KB total ukey num = 0.008494384 Billion total entry num = 0.008494384 Billion write speed all = 0.025686404 MB/sec (with seq num) write speed all = 0.024298539 MB/sec (without seq num) 2024/11/22-22:28:26.856420 140025171461696 [compaction_job.cc:1948] [default] [JOB 12764] Generated table #231330: 925147 keys, 37716704 bytes, temperature: kWarm 2024/11/22-22:28:26.856481 140025171461696 EVENT_LOG_v1 {"time_micros": 1732285706856451, "cf_name": "default", "job": 12764, "event": "table_file_creation", "file_number": 231330, "file_size": 37716704, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369194706, "largest_seqno": 3609091711, "table_properties": {"data_size": 31631424, "index_size": 1760256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25904116, "raw_average_key_size": 28, "raw_value_size": 108313650, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925147, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732285703, "oldest_key_time": 0, "file_creation_time": 1732285703, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 231330, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:28:26.893907 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231331.sst) /dev/shm/ToplingTemp/Topling-315261-YaZh8L 2024/11/22-22:28:27.681123 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231331.sst): first pass time = 0.79's, 170.504'MB/sec 2024/11/22-22:28:29.011517 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231331.sst): old prealloc_size = 147639500, real_size = 33576000 2024/11/22-22:28:29.012103 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231331.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924837 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 117.13 avg-zval = 34.15 2024/11/22-22:28:29.012869 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231331.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:28:29.957748 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231331.sst): second pass time = 1.33's, 81.497'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.932'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3129.065'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.283'MB/sec write SST data time = 0.13's, 294.289'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 37021.537'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.497'MB/sec zip pipeline throughput = 81.497'MB/sec entries = 924837 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.13 avg-zval = 34.15 usrkeys = 924837 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.13 avg-zval = 34.15 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924837 bytes = 3352544 } seq expand size = 6473859 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.5114 value = 3.4296 dict = 1.00 all = 3.5522 } Zip/UnZip{ index = 0.0951 value = 0.2916 dict = 1.00 all = 0.2815 } ---------------------------- total value len = 1.134656 GB avg = 0.120 KB (by entry num) total key len = 0.263738 GB avg = 0.028 KB total ukey len = 0.188384 GB avg = 0.020 KB total ukey num = 0.009419221 Billion total entry num = 0.009419221 Billion write speed all = 0.028557961 MB/sec (with seq num) write speed all = 0.027019088 MB/sec (without seq num) 2024/11/22-22:28:29.989486 140025171461696 [compaction_job.cc:1948] [default] [JOB 12764] Generated table #231331: 924837 keys, 37784398 bytes, temperature: kWarm 2024/11/22-22:28:29.989549 140025171461696 EVENT_LOG_v1 {"time_micros": 1732285709989518, "cf_name": "default", "job": 12764, "event": "table_file_creation", "file_number": 231331, "file_size": 37784398, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3326330223, "largest_seqno": 3609091731, "table_properties": {"data_size": 31584944, "index_size": 1759680, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25895436, "raw_average_key_size": 28, "raw_value_size": 108322330, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924837, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732285706, "oldest_key_time": 0, "file_creation_time": 1732285706, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 231331, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:28:30.027255 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231332.sst) /dev/shm/ToplingTemp/Topling-315261-Jyni0I 2024/11/22-22:28:30.924812 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231332.sst): first pass time = 0.90's, 149.544'MB/sec 2024/11/22-22:28:32.261461 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231332.sst): old prealloc_size = 147639500, real_size = 32934928 2024/11/22-22:28:32.262051 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231332.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930684 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.21 avg-zval = 33.24 2024/11/22-22:28:32.263386 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231332.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:28:33.201417 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231332.sst): second pass time = 1.34's, 80.989'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.287'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3082.444'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.288'MB/sec write SST data time = 0.13's, 283.606'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34010.220'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.989'MB/sec zip pipeline throughput = 80.989'MB/sec entries = 930684 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.21 avg-zval = 33.24 usrkeys = 930684 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.21 avg-zval = 33.24 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930684 bytes = 3141072 } seq expand size = 6514788 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5140 value = 3.4967 dict = 1.00 all = 3.6346 } Zip/UnZip{ index = 0.0951 value = 0.2860 dict = 1.00 all = 0.2751 } ---------------------------- total value len = 1.249329 GB avg = 0.121 KB (by entry num) total key len = 0.289797 GB avg = 0.028 KB total ukey len = 0.206998 GB avg = 0.020 KB total ukey num = 0.010349905 Billion total entry num = 0.010349905 Billion write speed all = 0.031429917 MB/sec (with seq num) write speed all = 0.029739105 MB/sec (without seq num) 2024/11/22-22:28:33.234256 140025171461696 [compaction_job.cc:1948] [default] [JOB 12764] Generated table #231332: 930684 keys, 36927360 bytes, temperature: kWarm 2024/11/22-22:28:33.234321 140025171461696 EVENT_LOG_v1 {"time_micros": 1732285713234288, "cf_name": "default", "job": 12764, "event": "table_file_creation", "file_number": 231332, "file_size": 36927360, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3515630317, "largest_seqno": 3609091759, "table_properties": {"data_size": 30931712, "index_size": 1770368, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26059152, "raw_average_key_size": 28, "raw_value_size": 108158664, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930684, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732285710, "oldest_key_time": 0, "file_creation_time": 1732285710, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 231332, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:28:33.270346 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231333.sst) /dev/shm/ToplingTemp/Topling-315261-4Y7rqM 2024/11/22-22:28:34.192965 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231333.sst): first pass time = 0.92's, 145.481'MB/sec 2024/11/22-22:28:35.533315 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231333.sst): old prealloc_size = 147639500, real_size = 33230080 2024/11/22-22:28:35.533874 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231333.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929029 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.47 avg-zval = 33.63 2024/11/22-22:28:35.534611 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231333.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:28:36.460274 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231333.sst): second pass time = 1.34's, 80.770'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.289'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3270.305'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.292'MB/sec write SST data time = 0.13's, 286.799'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 25800.363'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.770'MB/sec zip pipeline throughput = 80.770'MB/sec entries = 929029 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.47 avg-zval = 33.63 usrkeys = 929029 avg-key = 20.00 avg-zkey = 1.89 avg-val = 123.17 avg-zval = 33.63 TagRS{ kinds = RS_Key0_TagN bytes = 145280 } TagArray{ size = 889504 bytes = 3113280 } seq expand size = 6226528 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5849 value = 3.4634 dict = 1.00 all = 3.5939 } Zip/UnZip{ index = 0.0945 value = 0.2887 dict = 1.00 all = 0.2782 } ---------------------------- total value len = 1.363761 GB avg = 0.121 KB (by entry num) total key len = 0.315810 GB avg = 0.028 KB total ukey len = 0.225579 GB avg = 0.020 KB total ukey num = 0.011278934 Billion total entry num = 0.011278934 Billion write speed all = 0.034295596 MB/sec (with seq num) write speed all = 0.032453136 MB/sec (without seq num) 2024/11/22-22:28:36.493705 140025171461696 [compaction_job.cc:1948] [default] [JOB 12764] Generated table #231333: 929029 keys, 37345514 bytes, temperature: kWarm 2024/11/22-22:28:36.493763 140025171461696 EVENT_LOG_v1 {"time_micros": 1732285716493732, "cf_name": "default", "job": 12764, "event": "table_file_creation", "file_number": 231333, "file_size": 37345514, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3609091783, "table_properties": {"data_size": 31242224, "index_size": 1755392, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26012812, "raw_average_key_size": 28, "raw_value_size": 108205046, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929029, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732285713, "oldest_key_time": 0, "file_creation_time": 1732285713, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 231333, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:28:36.532761 140025171461696 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231334.sst) /dev/shm/ToplingTemp/Topling-315261-WxCw7K 2024/11/22-22:28:36.797997 140025171461696 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231334.sst): first pass time = 0.27's, 279.198'MB/sec 2024/11/22-22:28:37.556614 140025171461696 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231334.sst): old prealloc_size = 147639500, real_size = 20184272 2024/11/22-22:28:37.556978 140025171461696 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231334.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 511302 prefix = 4 raw-key = 0.0082 GB zip-key = 0.0010 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0597 GB zip-val = 0.0191 GB avg-val = 116.81 avg-zval = 37.32 2024/11/22-22:28:37.557023 140025171461696 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231334.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.074 GB 2024/11/22-22:28:37.905180 140025171461696 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00/231334.sst): second pass time = 0.76's, 78.796'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.35's, 59.274'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2848.268'MB/sec (index lex order gen) rebuild zvType time = 0.28's, 0.457'MB/sec write SST data time = 0.07's, 300.523'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.598'MB, throughput = 19945.975'MB/sec zip my value time = 0.76's, unzip length = 0.060'GB zip my value throughput = 78.796'MB/sec zip pipeline throughput = 78.796'MB/sec entries = 511302 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.81 avg-zval = 37.32 usrkeys = 511302 avg-key = 20.00 avg-zkey = 1.90 avg-val = 116.81 avg-zval = 37.32 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0102 GB value = 0.0597 GB dict = 0.60 MB all = 0.0740 GB } __ZipSize{ index = 0.0010 GB value = 0.0191 GB dict = 0.60 MB all = 0.0207 GB } UnZip/Zip{ index = 10.5016 value = 3.1298 dict = 1.00 all = 3.5837 } Zip/UnZip{ index = 0.0952 value = 0.3195 dict = 1.00 all = 0.2790 } ---------------------------- total value len = 1.423485 GB avg = 0.121 KB (by entry num) total key len = 0.330127 GB avg = 0.028 KB total ukey len = 0.235805 GB avg = 0.020 KB total ukey num = 0.011790236 Billion total entry num = 0.011790236 Billion write speed all = 0.035806403 MB/sec (with seq num) write speed all = 0.033880476 MB/sec (without seq num) 2024/11/22-22:28:37.926381 140025171461696 [compaction_job.cc:1948] [default] [JOB 12764] Generated table #231334: 511302 keys, 20660310 bytes, temperature: kWarm 2024/11/22-22:28:37.926443 140025171461696 EVENT_LOG_v1 {"time_micros": 1732285717926411, "cf_name": "default", "job": 12764, "event": "table_file_creation", "file_number": 231334, "file_size": 20660310, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 19082496, "index_size": 973760, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 14316456, "raw_average_key_size": 28, "raw_value_size": 59724576, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 511302, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732285716, "oldest_key_time": 0, "file_creation_time": 1732285716, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 231334, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:28:37.945112 140025171461696 [compaction_job.cc:703] job-12764: subcompact[0], size: 0.470282 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12764/att-00 [231322,231323,231324,231325,231326,231327,231328,231329,231330,231331,231332,231333,231334] 2024/11/22-22:28:38.029121 140025171461696 (Original Log Time 2024/11/22-22:27:57.092370) EVENT_LOG_v1 {"time_micros": 1732285677092312, "job": 12764, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [231253], "files_L3": [231318, 231319, 231222, 231223, 231184, 231185, 231186, 231187, 231188, 231189, 231190, 231198, 231199, 231283], "score": 0.999135, "input_data_size": 492537664, "oldest_snapshot_seqno": 3609627414} 2024/11/22-22:28:38.029125 140025171461696 (Original Log Time 2024/11/22-22:28:38.028385) [compaction_job.cc:2022] [default] [JOB 12764] Compacted 1@2 + 14@3 files to L3 => 470281603 bytes 2024/11/22-22:28:38.029126 140025171461696 (Original Log Time 2024/11/22-22:28:38.029056) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 13 0 0 0] max score 0.02, MB/sec: 12.1 rd, 11.5 wr, level 3, files in(1, 14) out(13 +0 blob) MB in(27.1, 442.6 +0.0 blob) out(448.5 +0.0 blob), read-write-amplify(33.9) write-amplify(16.6) OK, records in: 11869412, records dropped: 79176 output_compression: Snappy 2024/11/22-22:28:38.029132 140025171461696 (Original Log Time 2024/11/22-22:28:38.029100) EVENT_LOG_v1 {"time_micros": 1732285718029079, "cf": "default", "job": 12764, "event": "compaction_finished", "compaction_time_micros": 40852750, "compaction_time_cpu_micros": 18126357, "output_level": 3, "num_output_files": 13, "total_output_size": 470281603, "num_input_records": 11869412, "num_output_records": 11790236, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 294801167, "file_range_sync_nanos": 0, "file_fsync_nanos": 12095, "file_prepare_write_nanos": 30714, "lsm_state": [0, 0, 0, 13, 0, 0, 0]} 2024/11/22-22:28:38.153713 140025171461696 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732285676,"executesMs":41073,"compactionJobId":12764,"attempt":0,"compactionInputRawBytes":1695897705,"compactionInputZipBytes":492537664,"compactionOutputRawBytes":0,"compactionOutputZipBytes":470281603,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-af74f3a5dd0940eb9de0548921f34d84-1cf96371695a0ce8-00"}}, response = 2024/11/22-22:28:38.153790 140025171461696 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-12764/att-00: olev 3, work 41.074 s, result 4.949 ms, install 0.766 ms, input{raw 1.579 GiB zip 469.721 MiB}