2024/11/22-22:14:03.950504 140025213425216 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00: fork to child time = 0.023798 sec 2024/11/22-22:14:03.951532 140025213425216 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00 2024/11/22-22:14:03.960150 140025213425216 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00 2024/11/22-22:14:03.961590 140025213425216 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-22:14:03.961968 140025213425216 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-22:14:04.014937 140025213425216 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-12480/att-00/MANIFEST-206347 2024/11/22-22:14:04.037936 140025213425216 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-12480/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 230712, last_sequence is 3601773183, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 230530 2024/11/22-22:14:04.037948 140025213425216 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-22:14:04.038100 140025213425216 [version_set.cc:5721] Creating manifest 230712 2024/11/22-22:14:04.069981 140025213425216 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-22:14:04.091602 140025213425216 [compaction_job.cc:2437] [default] [JOB 12480] Compacting 1@2 + 15@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/22-22:14:04.091622 140025213425216 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 9389 Base level 2, inputs: [230672(27MB)], [230705(8812KB) 230603(34MB) 230604(35MB) 230605(35MB) 230606(36MB) 230607(34MB) 230621(35MB) 230622(35MB) 230623(35MB) 230624(36MB) 230625(36MB) 230626(33MB) 230627(35MB) 230628(35MB) 230706(34MB)] 2024/11/22-22:14:04.097531 140025213425216 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-22:14:04.097566 140025213425216 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-22:14:04.097586 140025213425216 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-22:14:04.097592 140025213425216 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-22:14:04.097604 140025213425216 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-22:14:04.369803 140025213425216 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230713.sst) /dev/shm/ToplingTemp/Topling-311198-XtIJDy 2024/11/22-22:14:05.479788 140025213425216 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230713.sst): first pass time = 1.11's, 120.924'MB/sec 2024/11/22-22:14:07.524355 140025213425216 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230713.sst): old prealloc_size = 147639500, real_size = 35152704 2024/11/22-22:14:07.525010 140025213425216 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230713.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1082995 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1039 GB zip-val = 0.0328 GB avg-val = 95.93 avg-zval = 30.28 2024/11/22-22:14:07.525241 140025213425216 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230713.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:14:08.394513 140025213425216 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230713.sst): second pass time = 2.04's, 50.836'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 41.733'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3281.053'MB/sec (index lex order gen) rebuild zvType time = 0.64's, 0.426'MB/sec write SST data time = 0.23's, 155.539'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.035'MB, throughput = 29378.380'MB/sec zip my value time = 2.04's, unzip length = 0.104'GB zip my value throughput = 50.836'MB/sec zip pipeline throughput = 130.263'MB/sec entries = 1082995 avg-key = 28.00 avg-zkey = 1.93 avg-val = 95.93 avg-zval = 30.28 usrkeys = 1082995 avg-key = 20.00 avg-zkey = 1.93 avg-val = 96.40 avg-zval = 30.28 TagRS{ kinds = RS_Key0_TagN bytes = 169320 } TagArray{ size = 72126 bytes = 216400 } seq expand size = 504882 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0217 GB value = 0.1039 GB dict = 1.04 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0328 GB dict = 1.04 MB all = 0.0363 GB } UnZip/Zip{ index = 10.3799 value = 3.1680 dict = 1.00 all = 3.6960 } Zip/UnZip{ index = 0.0963 value = 0.3157 dict = 1.00 all = 0.2706 } ---------------------------- total value len = 0.104399 GB avg = 0.096 KB (by entry num) total key len = 0.030324 GB avg = 0.028 KB total ukey len = 0.021660 GB avg = 0.020 KB total ukey num = 0.001082995 Billion total entry num = 0.001082995 Billion write speed all = 0.002800580 MB/sec (with seq num) write speed all = 0.002620476 MB/sec (without seq num) 2024/11/22-22:14:08.428506 140025213425216 [compaction_job.cc:1948] [default] [JOB 12480] Generated table #230713: 1082995 keys, 36314112 bytes, temperature: kWarm 2024/11/22-22:14:08.428592 140025213425216 EVENT_LOG_v1 {"time_micros": 1732284848428540, "cf_name": "default", "job": 12480, "event": "table_file_creation", "file_number": 230713, "file_size": 36314112, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3601542758, "table_properties": {"data_size": 32795056, "index_size": 2086720, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30323860, "raw_average_key_size": 28, "raw_value_size": 103894007, "raw_average_value_size": 95, "num_data_blocks": 1, "num_entries": 1082995, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732284844, "oldest_key_time": 0, "file_creation_time": 1732284844, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 230713, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:14:08.470062 140025213425216 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230714.sst) /dev/shm/ToplingTemp/Topling-311198-fjOuCy 2024/11/22-22:14:09.289409 140025213425216 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230714.sst): first pass time = 0.82's, 163.819'MB/sec 2024/11/22-22:14:11.032213 140025213425216 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230714.sst): old prealloc_size = 147639500, real_size = 35073248 2024/11/22-22:14:11.032722 140025213425216 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230714.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922403 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0331 GB avg-val = 117.51 avg-zval = 35.86 2024/11/22-22:14:11.033258 140025213425216 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230714.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:14:12.039801 140025213425216 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230714.sst): second pass time = 1.74's, 62.218'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.01's, 37.702'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3622.037'MB/sec (index lex order gen) rebuild zvType time = 0.88's, 0.263'MB/sec write SST data time = 0.13's, 293.500'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32280.615'MB/sec zip my value time = 1.74's, unzip length = 0.108'GB zip my value throughput = 62.218'MB/sec zip pipeline throughput = 69.566'MB/sec entries = 922403 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.51 avg-zval = 35.86 usrkeys = 922403 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.64 avg-zval = 35.86 TagRS{ kinds = RS_Key0_TagN bytes = 144240 } TagArray{ size = 544869 bytes = 1907056 } seq expand size = 3814083 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0331 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.4446 value = 3.2770 dict = 1.00 all = 3.5333 } Zip/UnZip{ index = 0.0957 value = 0.3052 dict = 1.00 all = 0.2830 } ---------------------------- total value len = 0.216604 GB avg = 0.108 KB (by entry num) total key len = 0.056151 GB avg = 0.028 KB total ukey len = 0.040108 GB avg = 0.020 KB total ukey num = 0.002005398 Billion total entry num = 0.002005398 Billion write speed all = 0.005669521 MB/sec (with seq num) write speed all = 0.005336045 MB/sec (without seq num) 2024/11/22-22:14:12.072786 140025213425216 [compaction_job.cc:1948] [default] [JOB 12480] Generated table #230714: 922403 keys, 37986841 bytes, temperature: kWarm 2024/11/22-22:14:12.072852 140025213425216 EVENT_LOG_v1 {"time_micros": 1732284852072818, "cf_name": "default", "job": 12480, "event": "table_file_creation", "file_number": 230714, "file_size": 37986841, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3486133321, "table_properties": {"data_size": 33076176, "index_size": 1766272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25827284, "raw_average_key_size": 28, "raw_value_size": 108390570, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922403, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732284848, "oldest_key_time": 0, "file_creation_time": 1732284848, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 230714, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:14:12.107437 140025213425216 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230715.sst) /dev/shm/ToplingTemp/Topling-311198-FwWZXu 2024/11/22-22:14:13.122492 140025213425216 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230715.sst): first pass time = 1.02's, 132.232'MB/sec 2024/11/22-22:14:14.739696 140025213425216 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230715.sst): old prealloc_size = 147639500, real_size = 33566752 2024/11/22-22:14:14.740371 140025213425216 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230715.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924824 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 117.13 avg-zval = 34.14 2024/11/22-22:14:14.741444 140025213425216 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230715.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:14:15.707221 140025213425216 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230715.sst): second pass time = 1.62's, 67.013'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.97's, 38.804'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2683.029'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.282'MB/sec write SST data time = 0.15's, 254.198'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 40396.623'MB/sec zip my value time = 1.62's, unzip length = 0.108'GB zip my value throughput = 67.013'MB/sec zip pipeline throughput = 67.013'MB/sec entries = 924824 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.13 avg-zval = 34.14 usrkeys = 924824 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.13 avg-zval = 34.14 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924824 bytes = 3121296 } seq expand size = 6473768 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5147 value = 3.4305 dict = 1.00 all = 3.5749 } Zip/UnZip{ index = 0.0951 value = 0.2915 dict = 1.00 all = 0.2797 } ---------------------------- total value len = 0.331400 GB avg = 0.113 KB (by entry num) total key len = 0.082046 GB avg = 0.028 KB total ukey len = 0.058604 GB avg = 0.020 KB total ukey num = 0.002930222 Billion total entry num = 0.002930222 Billion write speed all = 0.008593304 MB/sec (with seq num) write speed all = 0.008106076 MB/sec (without seq num) 2024/11/22-22:14:15.739651 140025213425216 [compaction_job.cc:1948] [default] [JOB 12480] Generated table #230715: 924824 keys, 37544130 bytes, temperature: kWarm 2024/11/22-22:14:15.739712 140025213425216 EVENT_LOG_v1 {"time_micros": 1732284855739681, "cf_name": "default", "job": 12480, "event": "table_file_creation", "file_number": 230715, "file_size": 37544130, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3486133322, "largest_seqno": 3601542784, "table_properties": {"data_size": 31576272, "index_size": 1759104, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25895072, "raw_average_key_size": 28, "raw_value_size": 108322748, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924824, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732284852, "oldest_key_time": 0, "file_creation_time": 1732284852, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 230715, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:14:15.776196 140025213425216 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230716.sst) /dev/shm/ToplingTemp/Topling-311198-ZLDhzy 2024/11/22-22:14:16.620586 140025213425216 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230716.sst): first pass time = 0.84's, 158.959'MB/sec 2024/11/22-22:14:18.042233 140025213425216 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230716.sst): old prealloc_size = 147639500, real_size = 33193088 2024/11/22-22:14:18.042891 140025213425216 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230716.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927891 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.65 avg-zval = 33.61 2024/11/22-22:14:18.044939 140025213425216 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230716.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:14:19.007478 140025213425216 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230716.sst): second pass time = 1.42's, 76.222'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.97's, 38.638'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2809.340'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.283'MB/sec write SST data time = 0.15's, 256.476'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32396.928'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 76.222'MB/sec zip pipeline throughput = 76.222'MB/sec entries = 927891 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.65 avg-zval = 33.61 usrkeys = 927891 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.65 avg-zval = 33.61 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927891 bytes = 3247632 } seq expand size = 6495237 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4440 value = 3.4709 dict = 1.00 all = 3.5989 } Zip/UnZip{ index = 0.0957 value = 0.2881 dict = 1.00 all = 0.2779 } ---------------------------- total value len = 0.446132 GB avg = 0.116 KB (by entry num) total key len = 0.108027 GB avg = 0.028 KB total ukey len = 0.077162 GB avg = 0.020 KB total ukey num = 0.003858113 Billion total entry num = 0.003858113 Billion write speed all = 0.011517175 MB/sec (with seq num) write speed all = 0.010875705 MB/sec (without seq num) 2024/11/22-22:14:19.043975 140025213425216 [compaction_job.cc:1948] [default] [JOB 12480] Generated table #230716: 927891 keys, 37294528 bytes, temperature: kWarm 2024/11/22-22:14:19.044045 140025213425216 EVENT_LOG_v1 {"time_micros": 1732284859044012, "cf_name": "default", "job": 12480, "event": "table_file_creation", "file_number": 230716, "file_size": 37294528, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3365258144, "largest_seqno": 3601542617, "table_properties": {"data_size": 31184048, "index_size": 1776896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25980948, "raw_average_key_size": 28, "raw_value_size": 108236902, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927891, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732284855, "oldest_key_time": 0, "file_creation_time": 1732284855, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 230716, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:14:19.081795 140025213425216 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230717.sst) /dev/shm/ToplingTemp/Topling-311198-aaQgax 2024/11/22-22:14:19.952564 140025213425216 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230717.sst): first pass time = 0.87's, 154.144'MB/sec 2024/11/22-22:14:21.425052 140025213425216 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230717.sst): old prealloc_size = 147639500, real_size = 32352768 2024/11/22-22:14:21.425566 140025213425216 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230717.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922906 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0304 GB avg-val = 117.43 avg-zval = 32.90 2024/11/22-22:14:21.426365 140025213425216 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230717.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:14:22.313849 140025213425216 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230717.sst): second pass time = 1.47's, 73.643'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 41.131'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3563.985'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.304'MB/sec write SST data time = 0.13's, 281.869'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 36514.702'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.643'MB/sec zip pipeline throughput = 73.643'MB/sec entries = 922906 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.43 avg-zval = 32.90 usrkeys = 922906 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.43 avg-zval = 32.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922906 bytes = 3345552 } seq expand size = 6460342 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5124 value = 3.5690 dict = 1.00 all = 3.6715 } Zip/UnZip{ index = 0.0951 value = 0.2802 dict = 1.00 all = 0.2724 } ---------------------------- total value len = 0.560969 GB avg = 0.117 KB (by entry num) total key len = 0.133869 GB avg = 0.028 KB total ukey len = 0.095620 GB avg = 0.020 KB total ukey num = 0.004781019 Billion total entry num = 0.004781019 Billion write speed all = 0.014439918 MB/sec (with seq num) write speed all = 0.013645055 MB/sec (without seq num) 2024/11/22-22:14:22.345053 140025213425216 [compaction_job.cc:1948] [default] [JOB 12480] Generated table #230717: 922906 keys, 36556304 bytes, temperature: kWarm 2024/11/22-22:14:22.345116 140025213425216 EVENT_LOG_v1 {"time_micros": 1732284862345084, "cf_name": "default", "job": 12480, "event": "table_file_creation", "file_number": 230717, "file_size": 36556304, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3601542636, "table_properties": {"data_size": 30366000, "index_size": 1755840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25841368, "raw_average_key_size": 28, "raw_value_size": 108376468, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922906, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732284859, "oldest_key_time": 0, "file_creation_time": 1732284859, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 230717, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:14:22.380679 140025213425216 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230718.sst) /dev/shm/ToplingTemp/Topling-311198-SYEf7w 2024/11/22-22:14:23.189664 140025213425216 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230718.sst): first pass time = 0.81's, 165.916'MB/sec 2024/11/22-22:14:24.657312 140025213425216 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230718.sst): old prealloc_size = 147639500, real_size = 33325856 2024/11/22-22:14:24.657960 140025213425216 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230718.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929759 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0313 GB avg-val = 116.36 avg-zval = 33.69 2024/11/22-22:14:24.659476 140025213425216 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230718.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:14:25.594729 140025213425216 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230718.sst): second pass time = 1.47's, 73.750'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.932'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2834.755'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.293'MB/sec write SST data time = 0.14's, 262.252'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 31842.149'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.750'MB/sec zip pipeline throughput = 73.750'MB/sec entries = 929759 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.36 avg-zval = 33.69 usrkeys = 929759 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.36 avg-zval = 33.69 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929759 bytes = 3254176 } seq expand size = 6508313 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5131 value = 3.4537 dict = 1.00 all = 3.5856 } Zip/UnZip{ index = 0.0951 value = 0.2895 dict = 1.00 all = 0.2789 } ---------------------------- total value len = 0.675662 GB avg = 0.118 KB (by entry num) total key len = 0.159902 GB avg = 0.028 KB total ukey len = 0.114216 GB avg = 0.020 KB total ukey num = 0.005710778 Billion total entry num = 0.005710778 Billion write speed all = 0.017363264 MB/sec (with seq num) write speed all = 0.016413891 MB/sec (without seq num) 2024/11/22-22:14:25.627849 140025213425216 [compaction_job.cc:1948] [default] [JOB 12480] Generated table #230718: 929759 keys, 37432236 bytes, temperature: kWarm 2024/11/22-22:14:25.627922 140025213425216 EVENT_LOG_v1 {"time_micros": 1732284865627880, "cf_name": "default", "job": 12480, "event": "table_file_creation", "file_number": 230718, "file_size": 37432236, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369194706, "largest_seqno": 3601542656, "table_properties": {"data_size": 31324496, "index_size": 1768768, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26033252, "raw_average_key_size": 28, "raw_value_size": 108184566, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929759, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732284862, "oldest_key_time": 0, "file_creation_time": 1732284862, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 230718, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:14:25.664652 140025213425216 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230719.sst) /dev/shm/ToplingTemp/Topling-311198-qWZJnw 2024/11/22-22:14:26.497058 140025213425216 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230719.sst): first pass time = 0.83's, 161.249'MB/sec 2024/11/22-22:14:27.858631 140025213425216 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230719.sst): old prealloc_size = 147639500, real_size = 32219184 2024/11/22-22:14:27.859282 140025213425216 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230719.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924752 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0302 GB avg-val = 117.14 avg-zval = 32.69 2024/11/22-22:14:27.860804 140025213425216 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230719.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:14:28.822730 140025213425216 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230719.sst): second pass time = 1.36's, 79.604'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 37.784'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2797.426'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.287'MB/sec write SST data time = 0.16's, 231.263'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34168.431'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.604'MB/sec zip pipeline throughput = 79.604'MB/sec entries = 924752 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.14 avg-zval = 32.69 usrkeys = 924752 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.14 avg-zval = 32.69 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924752 bytes = 3352240 } seq expand size = 6473264 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5120 value = 3.5835 dict = 1.00 all = 3.6845 } Zip/UnZip{ index = 0.0951 value = 0.2791 dict = 1.00 all = 0.2714 } ---------------------------- total value len = 0.790460 GB avg = 0.119 KB (by entry num) total key len = 0.185795 GB avg = 0.028 KB total ukey len = 0.132711 GB avg = 0.020 KB total ukey num = 0.006635530 Billion total entry num = 0.006635530 Billion write speed all = 0.020285506 MB/sec (with seq num) write speed all = 0.019182474 MB/sec (without seq num) 2024/11/22-22:14:28.857277 140025213425216 [compaction_job.cc:1948] [default] [JOB 12480] Generated table #230719: 924752 keys, 36427324 bytes, temperature: kWarm 2024/11/22-22:14:28.857354 140025213425216 EVENT_LOG_v1 {"time_micros": 1732284868857309, "cf_name": "default", "job": 12480, "event": "table_file_creation", "file_number": 230719, "file_size": 36427324, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3326330223, "largest_seqno": 3601542676, "table_properties": {"data_size": 30228384, "index_size": 1759424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25893056, "raw_average_key_size": 28, "raw_value_size": 108324744, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924752, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732284865, "oldest_key_time": 0, "file_creation_time": 1732284865, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 230719, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:14:28.895375 140025213425216 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230720.sst) /dev/shm/ToplingTemp/Topling-311198-OWvify 2024/11/22-22:14:29.787530 140025213425216 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230720.sst): first pass time = 0.89's, 150.449'MB/sec 2024/11/22-22:14:31.174287 140025213425216 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230720.sst): old prealloc_size = 147639500, real_size = 33166560 2024/11/22-22:14:31.174935 140025213425216 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230720.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924928 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 117.11 avg-zval = 33.71 2024/11/22-22:14:31.177752 140025213425216 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230720.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:14:32.215740 140025213425216 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230720.sst): second pass time = 1.39's, 78.203'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.04's, 35.665'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2803.282'MB/sec (index lex order gen) rebuild zvType time = 0.88's, 0.262'MB/sec write SST data time = 0.16's, 233.177'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33730.592'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 78.203'MB/sec zip pipeline throughput = 78.203'MB/sec entries = 924928 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.11 avg-zval = 33.71 usrkeys = 924928 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.11 avg-zval = 33.71 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924928 bytes = 3121648 } seq expand size = 6474496 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5140 value = 3.4745 dict = 1.00 all = 3.6135 } Zip/UnZip{ index = 0.0951 value = 0.2878 dict = 1.00 all = 0.2767 } ---------------------------- total value len = 0.905254 GB avg = 0.120 KB (by entry num) total key len = 0.211693 GB avg = 0.028 KB total ukey len = 0.151209 GB avg = 0.020 KB total ukey num = 0.007560458 Billion total entry num = 0.007560458 Billion write speed all = 0.023207304 MB/sec (with seq num) write speed all = 0.021950608 MB/sec (without seq num) 2024/11/22-22:14:32.249382 140025213425216 [compaction_job.cc:1948] [default] [JOB 12480] Generated table #230720: 924928 keys, 37143734 bytes, temperature: kWarm 2024/11/22-22:14:32.249448 140025213425216 EVENT_LOG_v1 {"time_micros": 1732284872249414, "cf_name": "default", "job": 12480, "event": "table_file_creation", "file_number": 230720, "file_size": 37143734, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3500532323, "largest_seqno": 3601542902, "table_properties": {"data_size": 31175696, "index_size": 1759424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25897984, "raw_average_key_size": 28, "raw_value_size": 108319876, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924928, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732284868, "oldest_key_time": 0, "file_creation_time": 1732284868, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 230720, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:14:32.286998 140025213425216 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230721.sst) /dev/shm/ToplingTemp/Topling-311198-WMo6Ew 2024/11/22-22:14:33.235432 140025213425216 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230721.sst): first pass time = 0.95's, 141.522'MB/sec 2024/11/22-22:14:34.682454 140025213425216 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230721.sst): old prealloc_size = 147639500, real_size = 34460288 2024/11/22-22:14:34.683061 140025213425216 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230721.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928591 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0325 GB avg-val = 116.54 avg-zval = 34.96 2024/11/22-22:14:34.685086 140025213425216 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230721.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:14:35.653835 140025213425216 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230721.sst): second pass time = 1.45's, 74.860'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.97's, 39.701'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3014.581'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.285'MB/sec write SST data time = 0.15's, 248.996'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32197.758'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.860'MB/sec zip pipeline throughput = 74.860'MB/sec entries = 928591 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.54 avg-zval = 34.96 usrkeys = 928591 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.54 avg-zval = 34.96 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928591 bytes = 3250080 } seq expand size = 6500137 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.5139 value = 3.3337 dict = 1.00 all = 3.4804 } Zip/UnZip{ index = 0.0951 value = 0.3000 dict = 1.00 all = 0.2873 } ---------------------------- total value len = 1.019972 GB avg = 0.120 KB (by entry num) total key len = 0.237693 GB avg = 0.028 KB total ukey len = 0.169781 GB avg = 0.020 KB total ukey num = 0.008489049 Billion total entry num = 0.008489049 Billion write speed all = 0.026129195 MB/sec (with seq num) write speed all = 0.024718250 MB/sec (without seq num) 2024/11/22-22:14:35.687284 140025213425216 [compaction_job.cc:1948] [default] [JOB 12480] Generated table #230721: 928591 keys, 38563832 bytes, temperature: kWarm 2024/11/22-22:14:35.687349 140025213425216 EVENT_LOG_v1 {"time_micros": 1732284875687316, "cf_name": "default", "job": 12480, "event": "table_file_creation", "file_number": 230721, "file_size": 38563832, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 3601542725, "table_properties": {"data_size": 32461552, "index_size": 1766400, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26000548, "raw_average_key_size": 28, "raw_value_size": 108217234, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928591, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732284872, "oldest_key_time": 0, "file_creation_time": 1732284872, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 230721, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:14:35.726281 140025213425216 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230722.sst) /dev/shm/ToplingTemp/Topling-311198-cmLz8y 2024/11/22-22:14:36.542816 140025213425216 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230722.sst): first pass time = 0.82's, 164.383'MB/sec 2024/11/22-22:14:38.169148 140025213425216 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230722.sst): old prealloc_size = 147639500, real_size = 36733728 2024/11/22-22:14:38.169716 140025213425216 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230722.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929333 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0347 GB avg-val = 116.42 avg-zval = 37.36 2024/11/22-22:14:38.170046 140025213425216 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230722.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:14:38.858886 140025213425216 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230722.sst): second pass time = 1.63's, 66.557'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.69's, 55.947'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3251.094'MB/sec (index lex order gen) rebuild zvType time = 0.56's, 0.411'MB/sec write SST data time = 0.12's, 309.924'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 31200.694'MB/sec zip my value time = 1.63's, unzip length = 0.108'GB zip my value throughput = 66.557'MB/sec zip pipeline throughput = 66.557'MB/sec entries = 929333 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.42 avg-zval = 37.36 usrkeys = 929333 avg-key = 20.00 avg-zkey = 1.91 avg-val = 118.19 avg-zval = 37.36 TagRS{ kinds = RS_Key0_TagN bytes = 145320 } TagArray{ size = 235034 bytes = 852016 } seq expand size = 1645238 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0347 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.4444 value = 3.1161 dict = 1.00 all = 3.4783 } Zip/UnZip{ index = 0.0957 value = 0.3209 dict = 1.00 all = 0.2875 } ---------------------------- total value len = 1.129813 GB avg = 0.120 KB (by entry num) total key len = 0.263715 GB avg = 0.028 KB total ukey len = 0.188368 GB avg = 0.020 KB total ukey num = 0.009418382 Billion total entry num = 0.009418382 Billion write speed all = 0.028949952 MB/sec (with seq num) write speed all = 0.027384648 MB/sec (without seq num) 2024/11/22-22:14:38.894887 140025213425216 [compaction_job.cc:1948] [default] [JOB 12480] Generated table #230722: 929333 keys, 38587535 bytes, temperature: kWarm 2024/11/22-22:14:38.894951 140025213425216 EVENT_LOG_v1 {"time_micros": 1732284878894920, "cf_name": "default", "job": 12480, "event": "table_file_creation", "file_number": 230722, "file_size": 38587535, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3601542800, "table_properties": {"data_size": 34721616, "index_size": 1779584, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26021324, "raw_average_key_size": 28, "raw_value_size": 108196430, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929333, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732284875, "oldest_key_time": 0, "file_creation_time": 1732284875, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 230722, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:14:38.934945 140025213425216 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230723.sst) /dev/shm/ToplingTemp/Topling-311198-xJzxtx 2024/11/22-22:14:39.747423 140025213425216 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230723.sst): first pass time = 0.81's, 165.203'MB/sec 2024/11/22-22:14:41.150375 140025213425216 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230723.sst): old prealloc_size = 147639500, real_size = 31268896 2024/11/22-22:14:41.150988 140025213425216 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230723.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1085 GB zip-val = 0.0293 GB avg-val = 118.00 avg-zval = 31.84 2024/11/22-22:14:41.151793 140025213425216 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230723.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:14:42.015174 140025213425216 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230723.sst): second pass time = 1.40's, 77.407'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 41.005'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3002.657'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.312'MB/sec write SST data time = 0.13's, 279.418'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 30354.175'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.407'MB/sec zip pipeline throughput = 77.407'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 31.84 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 31.84 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3332480 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.3764 value = 3.7065 dict = 1.00 all = 3.7850 } Zip/UnZip{ index = 0.0964 value = 0.2698 dict = 1.00 all = 0.2642 } ---------------------------- total value len = 1.244726 GB avg = 0.120 KB (by entry num) total key len = 0.289455 GB avg = 0.028 KB total ukey len = 0.206754 GB avg = 0.020 KB total ukey num = 0.010337682 Billion total entry num = 0.010337682 Billion write speed all = 0.031869866 MB/sec (with seq num) write speed all = 0.030151891 MB/sec (without seq num) 2024/11/22-22:14:42.055906 140025213425216 [compaction_job.cc:1948] [default] [JOB 12480] Generated table #230723: 919300 keys, 35460530 bytes, temperature: kWarm 2024/11/22-22:14:42.055971 140025213425216 EVENT_LOG_v1 {"time_micros": 1732284882055938, "cf_name": "default", "job": 12480, "event": "table_file_creation", "file_number": 230723, "file_size": 35460530, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3289124830, "largest_seqno": 3586184414, "table_properties": {"data_size": 29266960, "index_size": 1771904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732284878, "oldest_key_time": 0, "file_creation_time": 1732284878, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 230723, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:14:42.102845 140025213425216 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230724.sst) /dev/shm/ToplingTemp/Topling-311198-3Jhdpz 2024/11/22-22:14:42.939855 140025213425216 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230724.sst): first pass time = 0.84's, 160.362'MB/sec 2024/11/22-22:14:44.376974 140025213425216 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230724.sst): old prealloc_size = 147639500, real_size = 32934656 2024/11/22-22:14:44.377512 140025213425216 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230724.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927826 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.66 avg-zval = 33.34 2024/11/22-22:14:44.379345 140025213425216 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230724.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:14:45.325554 140025213425216 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230724.sst): second pass time = 1.44's, 75.357'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 38.922'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3449.416'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.283'MB/sec write SST data time = 0.13's, 286.588'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32161.887'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.357'MB/sec zip pipeline throughput = 75.357'MB/sec entries = 927826 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.66 avg-zval = 33.34 usrkeys = 927826 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.66 avg-zval = 33.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927826 bytes = 3131424 } seq expand size = 6494782 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5133 value = 3.4986 dict = 1.00 all = 3.6354 } Zip/UnZip{ index = 0.0951 value = 0.2858 dict = 1.00 all = 0.2751 } ---------------------------- total value len = 1.359459 GB avg = 0.121 KB (by entry num) total key len = 0.315434 GB avg = 0.028 KB total ukey len = 0.225310 GB avg = 0.020 KB total ukey num = 0.011265508 Billion total entry num = 0.011265508 Billion write speed all = 0.034790525 MB/sec (with seq num) write speed all = 0.032918487 MB/sec (without seq num) 2024/11/22-22:14:45.356965 140025213425216 [compaction_job.cc:1948] [default] [JOB 12480] Generated table #230724: 927826 keys, 36919916 bytes, temperature: kWarm 2024/11/22-22:14:45.357030 140025213425216 EVENT_LOG_v1 {"time_micros": 1732284885356996, "cf_name": "default", "job": 12480, "event": "table_file_creation", "file_number": 230724, "file_size": 36919916, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 3601540678, "table_properties": {"data_size": 30937456, "index_size": 1765056, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25979128, "raw_average_key_size": 28, "raw_value_size": 108238628, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927826, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732284882, "oldest_key_time": 0, "file_creation_time": 1732284882, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 230724, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:14:45.393675 140025213425216 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230725.sst) /dev/shm/ToplingTemp/Topling-311198-pgaESv 2024/11/22-22:14:46.214429 140025213425216 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230725.sst): first pass time = 0.82's, 163.538'MB/sec 2024/11/22-22:14:47.695752 140025213425216 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230725.sst): old prealloc_size = 147639500, real_size = 33726768 2024/11/22-22:14:47.696295 140025213425216 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230725.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927328 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0317 GB avg-val = 116.74 avg-zval = 34.22 2024/11/22-22:14:47.699156 140025213425216 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230725.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:14:48.617057 140025213425216 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230725.sst): second pass time = 1.48's, 73.152'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 41.059'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3472.825'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.293'MB/sec write SST data time = 0.13's, 289.937'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34870.287'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.152'MB/sec zip pipeline throughput = 73.152'MB/sec entries = 927328 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.74 avg-zval = 34.22 usrkeys = 927328 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.74 avg-zval = 34.22 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927328 bytes = 3245664 } seq expand size = 6491296 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.5137 value = 3.4116 dict = 1.00 all = 3.5482 } Zip/UnZip{ index = 0.0951 value = 0.2931 dict = 1.00 all = 0.2818 } ---------------------------- total value len = 1.474203 GB avg = 0.121 KB (by entry num) total key len = 0.341399 GB avg = 0.028 KB total ukey len = 0.243857 GB avg = 0.020 KB total ukey num = 0.012192836 Billion total entry num = 0.012192836 Billion write speed all = 0.037710726 MB/sec (with seq num) write speed all = 0.035684728 MB/sec (without seq num) 2024/11/22-22:14:48.649910 140025213425216 [compaction_job.cc:1948] [default] [JOB 12480] Generated table #230725: 927328 keys, 37826590 bytes, temperature: kWarm 2024/11/22-22:14:48.649979 140025213425216 EVENT_LOG_v1 {"time_micros": 1732284888649942, "cf_name": "default", "job": 12480, "event": "table_file_creation", "file_number": 230725, "file_size": 37826590, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3438862052, "largest_seqno": 3601542264, "table_properties": {"data_size": 31730720, "index_size": 1764032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25965184, "raw_average_key_size": 28, "raw_value_size": 108252608, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927328, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732284885, "oldest_key_time": 0, "file_creation_time": 1732284885, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 230725, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:14:48.692518 140025213425216 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230726.sst) /dev/shm/ToplingTemp/Topling-311198-1s0gfz 2024/11/22-22:14:49.564916 140025213425216 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230726.sst): first pass time = 0.87's, 153.857'MB/sec 2024/11/22-22:14:50.890920 140025213425216 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230726.sst): old prealloc_size = 147639500, real_size = 33513328 2024/11/22-22:14:50.891484 140025213425216 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230726.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924814 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 117.13 avg-zval = 34.09 2024/11/22-22:14:50.893301 140025213425216 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230726.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:14:51.834249 140025213425216 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230726.sst): second pass time = 1.33's, 81.730'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.744'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3238.758'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.284'MB/sec write SST data time = 0.13's, 292.123'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 30990.227'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.730'MB/sec zip pipeline throughput = 81.730'MB/sec entries = 924814 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.13 avg-zval = 34.09 usrkeys = 924814 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.13 avg-zval = 34.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924814 bytes = 3121264 } seq expand size = 6473698 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5154 value = 3.4363 dict = 1.00 all = 3.5800 } Zip/UnZip{ index = 0.0951 value = 0.2910 dict = 1.00 all = 0.2793 } ---------------------------- total value len = 1.589000 GB avg = 0.121 KB (by entry num) total key len = 0.367294 GB avg = 0.028 KB total ukey len = 0.262353 GB avg = 0.020 KB total ukey num = 0.013117650 Billion total entry num = 0.013117650 Billion write speed all = 0.040630226 MB/sec (with seq num) write speed all = 0.038450704 MB/sec (without seq num) 2024/11/22-22:14:51.871419 140025213425216 [compaction_job.cc:1948] [default] [JOB 12480] Generated table #230726: 924814 keys, 37490574 bytes, temperature: kWarm 2024/11/22-22:14:51.871486 140025213425216 EVENT_LOG_v1 {"time_micros": 1732284891871452, "cf_name": "default", "job": 12480, "event": "table_file_creation", "file_number": 230726, "file_size": 37490574, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3485447251, "largest_seqno": 3601542400, "table_properties": {"data_size": 31522976, "index_size": 1758976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25894792, "raw_average_key_size": 28, "raw_value_size": 108323080, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924814, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732284888, "oldest_key_time": 0, "file_creation_time": 1732284888, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 230726, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:14:51.906031 140025213425216 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230727.sst) /dev/shm/ToplingTemp/Topling-311198-NkRLzx 2024/11/22-22:14:52.115551 140025213425216 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230727.sst): first pass time = 0.21's, 234.895'MB/sec 2024/11/22-22:14:52.585650 140025213425216 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230727.sst): old prealloc_size = 147639500, real_size = 12719952 2024/11/22-22:14:52.585915 140025213425216 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230727.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 342312 prefix = 4 raw-key = 0.0055 GB zip-key = 0.0007 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0396 GB zip-val = 0.0120 GB avg-val = 115.75 avg-zval = 34.99 2024/11/22-22:14:52.586215 140025213425216 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230727.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0008 GB, waited 0.000 sec, Key+Value = 0.049 GB 2024/11/22-22:14:52.931043 140025213425216 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00/230727.sst): second pass time = 0.47's, 84.398'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.35's, 40.980'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2727.530'MB/sec (index lex order gen) rebuild zvType time = 0.30's, 0.288'MB/sec write SST data time = 0.05's, 295.827'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.405'MB, throughput = 13145.344'MB/sec zip my value time = 0.47's, unzip length = 0.040'GB zip my value throughput = 84.398'MB/sec zip pipeline throughput = 84.398'MB/sec entries = 342312 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.75 avg-zval = 34.99 usrkeys = 342312 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.75 avg-zval = 34.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 342312 bytes = 1112528 } seq expand size = 2396184 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0068 GB value = 0.0396 GB dict = 0.40 MB all = 0.0492 GB } __ZipSize{ index = 0.0007 GB value = 0.0120 GB dict = 0.40 MB all = 0.0142 GB } UnZip/Zip{ index = 10.4313 value = 3.3079 dict = 1.00 all = 3.4766 } Zip/UnZip{ index = 0.0959 value = 0.3023 dict = 1.00 all = 0.2876 } ---------------------------- total value len = 1.631017 GB avg = 0.121 KB (by entry num) total key len = 0.376879 GB avg = 0.028 KB total ukey len = 0.269199 GB avg = 0.020 KB total ukey num = 0.013459962 Billion total entry num = 0.013459962 Billion write speed all = 0.041701000 MB/sec (with seq num) write speed all = 0.039464653 MB/sec (without seq num) 2024/11/22-22:14:52.948824 140025213425216 [compaction_job.cc:1948] [default] [JOB 12480] Generated table #230727: 342312 keys, 14153575 bytes, temperature: kWarm 2024/11/22-22:14:52.948887 140025213425216 EVENT_LOG_v1 {"time_micros": 1732284892948855, "cf_name": "default", "job": 12480, "event": "table_file_creation", "file_number": 230727, "file_size": 14153575, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3567205802, "largest_seqno": 3601539717, "table_properties": {"data_size": 11977856, "index_size": 656320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 9584736, "raw_average_key_size": 28, "raw_value_size": 39621232, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 342312, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732284891, "oldest_key_time": 0, "file_creation_time": 1732284891, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 230727, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:14:52.963423 140025213425216 [compaction_job.cc:703] job-12480: subcompact[0], size: 0.535702 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12480/att-00 [230713,230714,230715,230716,230717,230718,230719,230720,230721,230722,230723,230724,230725,230726,230727] 2024/11/22-22:14:53.064195 140025213425216 (Original Log Time 2024/11/22-22:14:04.091685) EVENT_LOG_v1 {"time_micros": 1732284844091639, "job": 12480, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [230672], "files_L3": [230705, 230603, 230604, 230605, 230606, 230607, 230621, 230622, 230623, 230624, 230625, 230626, 230627, 230628, 230706], "score": 0.999199, "input_data_size": 555345919, "oldest_snapshot_seqno": 3601773040} 2024/11/22-22:14:53.064198 140025213425216 (Original Log Time 2024/11/22-22:14:53.063373) [compaction_job.cc:2022] [default] [JOB 12480] Compacted 1@2 + 15@3 files to L3 => 535701761 bytes 2024/11/22-22:14:53.064200 140025213425216 (Original Log Time 2024/11/22-22:14:53.064117) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 11.4 rd, 11.0 wr, level 3, files in(1, 15) out(15 +0 blob) MB in(27.1, 502.5 +0.0 blob) out(510.9 +0.0 blob), read-write-amplify(38.3) write-amplify(18.8) OK, records in: 13564417, records dropped: 104455 output_compression: Snappy 2024/11/22-22:14:53.064205 140025213425216 (Original Log Time 2024/11/22-22:14:53.064173) EVENT_LOG_v1 {"time_micros": 1732284893064142, "cf": "default", "job": 12480, "event": "compaction_finished", "compaction_time_micros": 48871747, "compaction_time_cpu_micros": 21665220, "output_level": 3, "num_output_files": 15, "total_output_size": 535701761, "num_input_records": 13564417, "num_output_records": 13459962, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 362981593, "file_range_sync_nanos": 0, "file_fsync_nanos": 14062, "file_prepare_write_nanos": 35190, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/22-22:14:53.194414 140025213425216 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732284843,"executesMs":49106,"compactionJobId":12480,"attempt":0,"compactionInputRawBytes":1943088039,"compactionInputZipBytes":555345919,"compactionOutputRawBytes":0,"compactionOutputZipBytes":535701761,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-b4e8887f0ab9da3ed266f4159aa66645-9e8a35dd4c1592ea-00"}}, response = 2024/11/22-22:14:53.194500 140025213425216 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-12480/att-00: olev 3, work 49.106 s, result 6.378 ms, install 0.853 ms, input{raw 1.810 GiB zip 529.619 MiB}