2024/11/24-01:28:05.803673 139957265520192 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00: fork to child time = 0.008081 sec 2024/11/24-01:28:05.804355 139957265520192 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00 2024/11/24-01:28:05.811173 139957265520192 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00 2024/11/24-01:28:05.812067 139957265520192 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-01:28:05.812310 139957265520192 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-01:28:05.870031 139957265520192 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-39346/att-00/MANIFEST-206347 2024/11/24-01:28:05.886969 139957265520192 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-39346/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 287274, last_sequence is 4372561640, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 287093 2024/11/24-01:28:05.886991 139957265520192 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-01:28:05.887097 139957265520192 [version_set.cc:5721] Creating manifest 287274 2024/11/24-01:28:05.925047 139957265520192 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-01:28:05.944951 139957265520192 [compaction_job.cc:2437] [default] [JOB 39346] Compacting 1@2 + 14@3 files to L3, score 0.72, subcompactions 1 : 1 2024/11/24-01:28:05.944964 139957265520192 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 29482 Base level 2, inputs: [287223(27MB)], [287256(35MB) 287257(3846KB) 287183(22MB) 287184(33MB) 287185(34MB) 287186(34MB) 287187(34MB) 287188(1652KB) 287153(35MB) 287154(35MB) 287155(35MB) 287156(35MB) 287157(35MB) 287258(35MB)] 2024/11/24-01:28:05.951626 139957265520192 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-01:28:05.951654 139957265520192 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-01:28:05.951669 139957265520192 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-01:28:05.951674 139957265520192 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-01:28:05.951684 139957265520192 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-01:28:06.653277 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287275.sst) /dev/shm/ToplingTemp/Topling-237669-PrU6vB 2024/11/24-01:28:07.286598 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287275.sst): first pass time = 0.63's, 211.948'MB/sec 2024/11/24-01:28:08.641210 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287275.sst): old prealloc_size = 147639500, real_size = 32875376 2024/11/24-01:28:08.641745 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287275.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925809 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 116.97 avg-zval = 33.36 2024/11/24-01:28:08.642532 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287275.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:28:09.488991 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287275.sst): second pass time = 1.35's, 80.005'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.473'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3474.733'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 287.803'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35738.000'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.005'MB/sec zip pipeline throughput = 80.005'MB/sec entries = 925809 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.97 avg-zval = 33.36 usrkeys = 925809 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.97 avg-zval = 33.36 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925809 bytes = 3124624 } seq expand size = 6480663 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5121 value = 3.5067 dict = 1.00 all = 3.6418 } Zip/UnZip{ index = 0.0951 value = 0.2852 dict = 1.00 all = 0.2746 } ---------------------------- total value len = 8.963396 GB avg = 0.168 KB (by entry num) total key len = 1.466744 GB avg = 0.028 KB total ukey len = 1.016394 GB avg = 0.015 KB total ukey num = 0.066476244 Billion total entry num = 0.053283973 Billion write speed all = 0.175069093 MB/sec (with seq num) write speed all = 0.167914153 MB/sec (without seq num) 2024/11/24-01:28:09.537205 139957265520192 [compaction_job.cc:1948] [default] [JOB 39346] Generated table #287275: 925809 keys, 36854740 bytes, temperature: kWarm 2024/11/24-01:28:09.537307 139957265520192 EVENT_LOG_v1 {"time_micros": 1732382889537245, "cf_name": "default", "job": 39346, "event": "table_file_creation", "file_number": 287275, "file_size": 36854740, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178599, "largest_seqno": 4372193454, "table_properties": {"data_size": 30882336, "index_size": 1761408, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25922652, "raw_average_key_size": 28, "raw_value_size": 108295178, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925809, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732382886, "oldest_key_time": 0, "file_creation_time": 1732382886, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287275, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:28:09.580763 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287276.sst) /dev/shm/ToplingTemp/Topling-237669-CGCC7C 2024/11/24-01:28:10.817081 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287276.sst): first pass time = 1.24's, 81.392'MB/sec 2024/11/24-01:28:11.813190 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287276.sst): old prealloc_size = 147639500, real_size = 24048432 2024/11/24-01:28:11.813617 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287276.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 695739 prefix = 4 raw-key = 0.0111 GB zip-key = 0.0013 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0811 GB zip-val = 0.0225 GB avg-val = 116.63 avg-zval = 32.40 2024/11/24-01:28:11.814219 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287276.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0017 GB, waited 0.000 sec, Key+Value = 0.101 GB 2024/11/24-01:28:12.456427 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287276.sst): second pass time = 1.00's, 81.526'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 42.037'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3317.388'MB/sec (index lex order gen) rebuild zvType time = 0.55's, 0.319'MB/sec write SST data time = 0.10's, 277.575'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.813'MB, throughput = 20152.420'MB/sec zip my value time = 1.00's, unzip length = 0.081'GB zip my value throughput = 81.526'MB/sec zip pipeline throughput = 81.526'MB/sec entries = 695739 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.63 avg-zval = 32.40 usrkeys = 695739 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.63 avg-zval = 32.40 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 695739 bytes = 2348128 } seq expand size = 4870173 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0139 GB value = 0.0811 GB dict = 0.81 MB all = 0.1006 GB } __ZipSize{ index = 0.0013 GB value = 0.0225 GB dict = 0.81 MB all = 0.0270 GB } UnZip/Zip{ index = 10.4418 value = 3.5997 dict = 1.00 all = 3.7214 } Zip/UnZip{ index = 0.0958 value = 0.2778 dict = 1.00 all = 0.2687 } ---------------------------- total value len = 9.049409 GB avg = 0.168 KB (by entry num) total key len = 1.486224 GB avg = 0.028 KB total ukey len = 1.030309 GB avg = 0.015 KB total ukey num = 0.067171983 Billion total entry num = 0.053979712 Billion write speed all = 0.176830991 MB/sec (with seq num) write speed all = 0.169582990 MB/sec (without seq num) 2024/11/24-01:28:12.492200 139957265520192 [compaction_job.cc:1948] [default] [JOB 39346] Generated table #287276: 695739 keys, 27039066 bytes, temperature: kWarm 2024/11/24-01:28:12.492289 139957265520192 EVENT_LOG_v1 {"time_micros": 1732382892492236, "cf_name": "default", "job": 39346, "event": "table_file_creation", "file_number": 287276, "file_size": 27039066, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4296313160, "largest_seqno": 4372194224, "table_properties": {"data_size": 22541728, "index_size": 1332608, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 19480692, "raw_average_key_size": 28, "raw_value_size": 81142994, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 695739, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732382889, "oldest_key_time": 0, "file_creation_time": 1732382889, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287276, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:28:12.525837 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287277.sst) /dev/shm/ToplingTemp/Topling-237669-wjmgqB 2024/11/24-01:28:13.077767 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287277.sst): first pass time = 0.55's, 243.198'MB/sec 2024/11/24-01:28:14.358108 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287277.sst): old prealloc_size = 147639500, real_size = 31088816 2024/11/24-01:28:14.358685 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287277.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925932 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0291 GB avg-val = 116.95 avg-zval = 31.42 2024/11/24-01:28:14.360313 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287277.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:28:15.175115 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287277.sst): second pass time = 1.28's, 84.645'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.925'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3251.110'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.338'MB/sec write SST data time = 0.13's, 267.952'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32192.017'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.645'MB/sec zip pipeline throughput = 84.645'MB/sec entries = 925932 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.95 avg-zval = 31.42 usrkeys = 925932 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.95 avg-zval = 31.42 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925932 bytes = 3125040 } seq expand size = 6481524 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.5139 value = 3.7219 dict = 1.00 all = 3.8273 } Zip/UnZip{ index = 0.0951 value = 0.2687 dict = 1.00 all = 0.2613 } ---------------------------- total value len = 9.164182 GB avg = 0.167 KB (by entry num) total key len = 1.512151 GB avg = 0.028 KB total ukey len = 1.048828 GB avg = 0.015 KB total ukey num = 0.068097915 Billion total entry num = 0.054905644 Billion write speed all = 0.179184325 MB/sec (with seq num) write speed all = 0.171812333 MB/sec (without seq num) 2024/11/24-01:28:15.218585 139957265520192 [compaction_job.cc:1948] [default] [JOB 39346] Generated table #287277: 925932 keys, 35068862 bytes, temperature: kWarm 2024/11/24-01:28:15.218656 139957265520192 EVENT_LOG_v1 {"time_micros": 1732382895218621, "cf_name": "default", "job": 39346, "event": "table_file_creation", "file_number": 287277, "file_size": 35068862, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178645, "largest_seqno": 4372193177, "table_properties": {"data_size": 29095776, "index_size": 1761344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25926096, "raw_average_key_size": 28, "raw_value_size": 108291736, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925932, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732382892, "oldest_key_time": 0, "file_creation_time": 1732382892, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287277, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:28:15.260726 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287278.sst) /dev/shm/ToplingTemp/Topling-237669-AdzzcB 2024/11/24-01:28:16.254879 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287278.sst): first pass time = 0.99's, 135.012'MB/sec 2024/11/24-01:28:17.589279 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287278.sst): old prealloc_size = 147639500, real_size = 31507856 2024/11/24-01:28:17.589817 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287278.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925624 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0295 GB avg-val = 117.00 avg-zval = 31.89 2024/11/24-01:28:17.591449 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287278.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:28:18.420789 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287278.sst): second pass time = 1.33's, 81.211'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.679'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3439.180'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 273.091'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31779.484'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.211'MB/sec zip pipeline throughput = 81.211'MB/sec entries = 925624 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.00 avg-zval = 31.89 usrkeys = 925624 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.00 avg-zval = 31.89 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925624 bytes = 3124000 } seq expand size = 6479368 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.5127 value = 3.6693 dict = 1.00 all = 3.7822 } Zip/UnZip{ index = 0.0951 value = 0.2725 dict = 1.00 all = 0.2644 } ---------------------------- total value len = 9.278962 GB avg = 0.166 KB (by entry num) total key len = 1.538068 GB avg = 0.028 KB total ukey len = 1.067340 GB avg = 0.015 KB total ukey num = 0.069023539 Billion total entry num = 0.055831268 Billion write speed all = 0.181535803 MB/sec (with seq num) write speed all = 0.174039938 MB/sec (without seq num) 2024/11/24-01:28:18.466561 139957265520192 [compaction_job.cc:1948] [default] [JOB 39346] Generated table #287278: 925624 keys, 35486854 bytes, temperature: kWarm 2024/11/24-01:28:18.466637 139957265520192 EVENT_LOG_v1 {"time_micros": 1732382898466597, "cf_name": "default", "job": 39346, "event": "table_file_creation", "file_number": 287278, "file_size": 35486854, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178675, "largest_seqno": 4372193193, "table_properties": {"data_size": 29515328, "index_size": 1760960, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25917472, "raw_average_key_size": 28, "raw_value_size": 108300356, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925624, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732382895, "oldest_key_time": 0, "file_creation_time": 1732382895, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287278, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:28:18.503865 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287279.sst) /dev/shm/ToplingTemp/Topling-237669-bT0g2D 2024/11/24-01:28:19.527576 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287279.sst): first pass time = 1.02's, 131.114'MB/sec 2024/11/24-01:28:20.857260 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287279.sst): old prealloc_size = 147639500, real_size = 31847344 2024/11/24-01:28:20.857786 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287279.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925773 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0299 GB avg-val = 116.98 avg-zval = 32.25 2024/11/24-01:28:20.859420 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287279.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:28:21.681603 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287279.sst): second pass time = 1.33's, 81.520'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.462'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3498.968'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.13's, 277.395'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35627.663'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.520'MB/sec zip pipeline throughput = 81.520'MB/sec entries = 925773 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.98 avg-zval = 32.25 usrkeys = 925773 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.98 avg-zval = 32.25 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925773 bytes = 3124496 } seq expand size = 6480411 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5102 value = 3.6275 dict = 1.00 all = 3.7463 } Zip/UnZip{ index = 0.0951 value = 0.2757 dict = 1.00 all = 0.2669 } ---------------------------- total value len = 9.393738 GB avg = 0.166 KB (by entry num) total key len = 1.563990 GB avg = 0.028 KB total ukey len = 1.085856 GB avg = 0.016 KB total ukey num = 0.069949312 Billion total entry num = 0.056757041 Billion write speed all = 0.183886994 MB/sec (with seq num) write speed all = 0.176267253 MB/sec (without seq num) 2024/11/24-01:28:21.726589 139957265520192 [compaction_job.cc:1948] [default] [JOB 39346] Generated table #287279: 925773 keys, 35826558 bytes, temperature: kWarm 2024/11/24-01:28:21.726662 139957265520192 EVENT_LOG_v1 {"time_micros": 1732382901726626, "cf_name": "default", "job": 39346, "event": "table_file_creation", "file_number": 287279, "file_size": 35826558, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178686, "largest_seqno": 4372193204, "table_properties": {"data_size": 29854048, "index_size": 1761664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25921644, "raw_average_key_size": 28, "raw_value_size": 108296130, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925773, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732382898, "oldest_key_time": 0, "file_creation_time": 1732382898, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287279, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:28:21.768239 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287280.sst) /dev/shm/ToplingTemp/Topling-237669-ul9abE 2024/11/24-01:28:22.709320 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287280.sst): first pass time = 0.94's, 142.628'MB/sec 2024/11/24-01:28:24.048288 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287280.sst): old prealloc_size = 147639500, real_size = 32099856 2024/11/24-01:28:24.048828 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287280.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925817 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0301 GB avg-val = 116.97 avg-zval = 32.52 2024/11/24-01:28:24.049602 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287280.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:28:24.877866 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287280.sst): second pass time = 1.34's, 80.931'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.493'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3403.371'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 281.182'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 36029.540'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.931'MB/sec zip pipeline throughput = 80.931'MB/sec entries = 925817 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.97 avg-zval = 32.52 usrkeys = 925817 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.97 avg-zval = 32.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925817 bytes = 3124640 } seq expand size = 6480719 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.5122 value = 3.5970 dict = 1.00 all = 3.7201 } Zip/UnZip{ index = 0.0951 value = 0.2780 dict = 1.00 all = 0.2688 } ---------------------------- total value len = 9.508514 GB avg = 0.165 KB (by entry num) total key len = 1.589913 GB avg = 0.028 KB total ukey len = 1.104372 GB avg = 0.016 KB total ukey num = 0.070875129 Billion total entry num = 0.057682858 Billion write speed all = 0.186238136 MB/sec (with seq num) write speed all = 0.178494518 MB/sec (without seq num) 2024/11/24-01:28:24.922813 139957265520192 [compaction_job.cc:1948] [default] [JOB 39346] Generated table #287280: 925817 keys, 36079498 bytes, temperature: kWarm 2024/11/24-01:28:24.922884 139957265520192 EVENT_LOG_v1 {"time_micros": 1732382904922848, "cf_name": "default", "job": 39346, "event": "table_file_creation", "file_number": 287280, "file_size": 36079498, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178700, "largest_seqno": 4372193214, "table_properties": {"data_size": 30106816, "index_size": 1761408, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25922876, "raw_average_key_size": 28, "raw_value_size": 108294986, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925817, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732382901, "oldest_key_time": 0, "file_creation_time": 1732382901, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287280, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:28:24.969351 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287281.sst) /dev/shm/ToplingTemp/Topling-237669-ONPJRA 2024/11/24-01:28:26.021673 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287281.sst): first pass time = 1.05's, 127.549'MB/sec 2024/11/24-01:28:27.360184 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287281.sst): old prealloc_size = 147639500, real_size = 32303456 2024/11/24-01:28:27.360716 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287281.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925793 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0303 GB avg-val = 116.98 avg-zval = 32.74 2024/11/24-01:28:27.361503 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287281.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:28:28.197327 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287281.sst): second pass time = 1.34's, 80.963'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.342'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3479.121'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 280.568'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34933.872'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.963'MB/sec zip pipeline throughput = 80.963'MB/sec entries = 925793 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.98 avg-zval = 32.74 usrkeys = 925793 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.98 avg-zval = 32.74 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925793 bytes = 3124560 } seq expand size = 6480551 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.5135 value = 3.5729 dict = 1.00 all = 3.6992 } Zip/UnZip{ index = 0.0951 value = 0.2799 dict = 1.00 all = 0.2703 } ---------------------------- total value len = 9.623290 GB avg = 0.164 KB (by entry num) total key len = 1.615835 GB avg = 0.028 KB total ukey len = 1.122888 GB avg = 0.016 KB total ukey num = 0.071800922 Billion total entry num = 0.058608651 Billion write speed all = 0.188588634 MB/sec (with seq num) write speed all = 0.180721171 MB/sec (without seq num) 2024/11/24-01:28:28.242573 139957265520192 [compaction_job.cc:1948] [default] [JOB 39346] Generated table #287281: 925793 keys, 36282714 bytes, temperature: kWarm 2024/11/24-01:28:28.242649 139957265520192 EVENT_LOG_v1 {"time_micros": 1732382908242611, "cf_name": "default", "job": 39346, "event": "table_file_creation", "file_number": 287281, "file_size": 36282714, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178719, "largest_seqno": 4372193810, "table_properties": {"data_size": 30310672, "index_size": 1761152, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25922204, "raw_average_key_size": 28, "raw_value_size": 108295662, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925793, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732382904, "oldest_key_time": 0, "file_creation_time": 1732382904, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287281, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:28:28.279071 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287282.sst) /dev/shm/ToplingTemp/Topling-237669-SQ2b3B 2024/11/24-01:28:29.311227 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287282.sst): first pass time = 1.03's, 130.041'MB/sec 2024/11/24-01:28:30.633292 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287282.sst): old prealloc_size = 147639500, real_size = 32553600 2024/11/24-01:28:30.633845 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287282.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926071 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0306 GB avg-val = 116.93 avg-zval = 33.00 2024/11/24-01:28:30.635685 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287282.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:28:31.478642 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287282.sst): second pass time = 1.32's, 82.007'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.219'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3310.129'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.13's, 276.388'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34665.277'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.007'MB/sec zip pipeline throughput = 82.007'MB/sec entries = 926071 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.93 avg-zval = 33.00 usrkeys = 926071 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.93 avg-zval = 33.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926071 bytes = 3125504 } seq expand size = 6482497 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5113 value = 3.5435 dict = 1.00 all = 3.6738 } Zip/UnZip{ index = 0.0951 value = 0.2822 dict = 1.00 all = 0.2722 } ---------------------------- total value len = 9.738061 GB avg = 0.164 KB (by entry num) total key len = 1.641765 GB avg = 0.028 KB total ukey len = 1.141409 GB avg = 0.016 KB total ukey num = 0.072726993 Billion total entry num = 0.059534722 Billion write speed all = 0.190939024 MB/sec (with seq num) write speed all = 0.182947688 MB/sec (without seq num) 2024/11/24-01:28:31.523041 139957265520192 [compaction_job.cc:1948] [default] [JOB 39346] Generated table #287282: 926071 keys, 36534042 bytes, temperature: kWarm 2024/11/24-01:28:31.523114 139957265520192 EVENT_LOG_v1 {"time_micros": 1732382911523078, "cf_name": "default", "job": 39346, "event": "table_file_creation", "file_number": 287282, "file_size": 36534042, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178730, "largest_seqno": 4372194470, "table_properties": {"data_size": 30559856, "index_size": 1762048, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25929988, "raw_average_key_size": 28, "raw_value_size": 108287882, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926071, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732382908, "oldest_key_time": 0, "file_creation_time": 1732382908, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287282, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:28:31.571168 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287283.sst) /dev/shm/ToplingTemp/Topling-237669-YoVrUC 2024/11/24-01:28:32.609421 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287283.sst): first pass time = 1.04's, 129.278'MB/sec 2024/11/24-01:28:33.940969 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287283.sst): old prealloc_size = 147639500, real_size = 32581584 2024/11/24-01:28:33.941538 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287283.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925849 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0306 GB avg-val = 116.97 avg-zval = 33.04 2024/11/24-01:28:33.943244 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287283.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:28:34.783380 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287283.sst): second pass time = 1.33's, 81.375'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.402'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3354.715'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 278.679'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34149.475'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.375'MB/sec zip pipeline throughput = 81.375'MB/sec entries = 925849 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.97 avg-zval = 33.04 usrkeys = 925849 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.97 avg-zval = 33.04 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925849 bytes = 3124752 } seq expand size = 6480943 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5145 value = 3.5403 dict = 1.00 all = 3.6710 } Zip/UnZip{ index = 0.0951 value = 0.2825 dict = 1.00 all = 0.2724 } ---------------------------- total value len = 9.852836 GB avg = 0.163 KB (by entry num) total key len = 1.667688 GB avg = 0.028 KB total ukey len = 1.159926 GB avg = 0.016 KB total ukey num = 0.073652842 Billion total entry num = 0.060460571 Billion write speed all = 0.193289054 MB/sec (with seq num) write speed all = 0.185173891 MB/sec (without seq num) 2024/11/24-01:28:34.827936 139957265520192 [compaction_job.cc:1948] [default] [JOB 39346] Generated table #287283: 925849 keys, 36561346 bytes, temperature: kWarm 2024/11/24-01:28:34.828019 139957265520192 EVENT_LOG_v1 {"time_micros": 1732382914827972, "cf_name": "default", "job": 39346, "event": "table_file_creation", "file_number": 287283, "file_size": 36561346, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680186, "largest_seqno": 4372194412, "table_properties": {"data_size": 30588864, "index_size": 1761088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25923772, "raw_average_key_size": 28, "raw_value_size": 108294066, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925849, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732382911, "oldest_key_time": 0, "file_creation_time": 1732382911, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287283, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:28:34.864254 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287284.sst) /dev/shm/ToplingTemp/Topling-237669-p6oPqE 2024/11/24-01:28:35.709115 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287284.sst): first pass time = 0.84's, 158.871'MB/sec 2024/11/24-01:28:37.020645 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287284.sst): old prealloc_size = 147639500, real_size = 32871872 2024/11/24-01:28:37.021237 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287284.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925848 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 116.97 avg-zval = 33.35 2024/11/24-01:28:37.022018 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287284.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:28:37.867873 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287284.sst): second pass time = 1.31's, 82.622'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.498'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3092.338'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 285.329'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34064.238'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.622'MB/sec zip pipeline throughput = 82.622'MB/sec entries = 925848 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.97 avg-zval = 33.35 usrkeys = 925848 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.97 avg-zval = 33.35 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925848 bytes = 3124752 } seq expand size = 6480936 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5114 value = 3.5071 dict = 1.00 all = 3.6421 } Zip/UnZip{ index = 0.0951 value = 0.2851 dict = 1.00 all = 0.2746 } ---------------------------- total value len = 9.967611 GB avg = 0.162 KB (by entry num) total key len = 1.693612 GB avg = 0.028 KB total ukey len = 1.178443 GB avg = 0.016 KB total ukey num = 0.074578690 Billion total entry num = 0.061386419 Billion write speed all = 0.195639545 MB/sec (with seq num) write speed all = 0.187400539 MB/sec (without seq num) 2024/11/24-01:28:37.914247 139957265520192 [compaction_job.cc:1948] [default] [JOB 39346] Generated table #287284: 925848 keys, 36851746 bytes, temperature: kWarm 2024/11/24-01:28:37.914321 139957265520192 EVENT_LOG_v1 {"time_micros": 1732382917914283, "cf_name": "default", "job": 39346, "event": "table_file_creation", "file_number": 287284, "file_size": 36851746, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680199, "largest_seqno": 4372194107, "table_properties": {"data_size": 30878640, "index_size": 1761600, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25923744, "raw_average_key_size": 28, "raw_value_size": 108294104, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925848, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732382914, "oldest_key_time": 0, "file_creation_time": 1732382914, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287284, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:28:37.957908 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287285.sst) /dev/shm/ToplingTemp/Topling-237669-CoK23A 2024/11/24-01:28:38.985737 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287285.sst): first pass time = 1.03's, 130.589'MB/sec 2024/11/24-01:28:40.268015 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287285.sst): old prealloc_size = 147639500, real_size = 33262912 2024/11/24-01:28:40.268547 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287285.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925681 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.99 avg-zval = 33.77 2024/11/24-01:28:40.270502 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287285.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:28:41.123179 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287285.sst): second pass time = 1.28's, 84.531'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.550'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3493.724'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.319'MB/sec write SST data time = 0.13's, 286.428'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34588.073'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.531'MB/sec zip pipeline throughput = 84.531'MB/sec entries = 925681 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.99 avg-zval = 33.77 usrkeys = 925681 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.99 avg-zval = 33.77 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925681 bytes = 3124192 } seq expand size = 6479767 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.4458 value = 3.4646 dict = 1.00 all = 3.6039 } Zip/UnZip{ index = 0.0957 value = 0.2886 dict = 1.00 all = 0.2775 } ---------------------------- total value len = 10.082389 GB avg = 0.162 KB (by entry num) total key len = 1.719531 GB avg = 0.028 KB total ukey len = 1.196957 GB avg = 0.016 KB total ukey num = 0.075504371 Billion total entry num = 0.062312100 Billion write speed all = 0.197989206 MB/sec (with seq num) write speed all = 0.189626416 MB/sec (without seq num) 2024/11/24-01:28:41.169499 139957265520192 [compaction_job.cc:1948] [default] [JOB 39346] Generated table #287285: 925681 keys, 37241926 bytes, temperature: kWarm 2024/11/24-01:28:41.169570 139957265520192 EVENT_LOG_v1 {"time_micros": 1732382921169535, "cf_name": "default", "job": 39346, "event": "table_file_creation", "file_number": 287285, "file_size": 37241926, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680212, "largest_seqno": 4372194200, "table_properties": {"data_size": 31258928, "index_size": 1772352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25919068, "raw_average_key_size": 28, "raw_value_size": 108298758, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925681, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732382917, "oldest_key_time": 0, "file_creation_time": 1732382917, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287285, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:28:41.208870 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287286.sst) /dev/shm/ToplingTemp/Topling-237669-GGtj3C 2024/11/24-01:28:42.237246 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287286.sst): first pass time = 1.03's, 130.519'MB/sec 2024/11/24-01:28:43.573972 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287286.sst): old prealloc_size = 147639500, real_size = 33602256 2024/11/24-01:28:43.574514 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287286.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925961 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 116.95 avg-zval = 34.12 2024/11/24-01:28:43.575293 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287286.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:28:44.420588 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287286.sst): second pass time = 1.34's, 81.062'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.397'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3555.470'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 292.565'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33569.733'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.062'MB/sec zip pipeline throughput = 81.062'MB/sec entries = 925961 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.95 avg-zval = 34.12 usrkeys = 925961 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.95 avg-zval = 34.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925961 bytes = 3125136 } seq expand size = 6481727 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.4444 value = 3.4272 dict = 1.00 all = 3.5710 } Zip/UnZip{ index = 0.0957 value = 0.2918 dict = 1.00 all = 0.2800 } ---------------------------- total value len = 10.197162 GB avg = 0.161 KB (by entry num) total key len = 1.745458 GB avg = 0.028 KB total ukey len = 1.215476 GB avg = 0.016 KB total ukey num = 0.076430332 Billion total entry num = 0.063238061 Billion write speed all = 0.200338501 MB/sec (with seq num) write speed all = 0.191851909 MB/sec (without seq num) 2024/11/24-01:28:44.466675 139957265520192 [compaction_job.cc:1948] [default] [JOB 39346] Generated table #287286: 925961 keys, 37585912 bytes, temperature: kWarm 2024/11/24-01:28:44.466747 139957265520192 EVENT_LOG_v1 {"time_micros": 1732382924466711, "cf_name": "default", "job": 39346, "event": "table_file_creation", "file_number": 287286, "file_size": 37585912, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4372194250, "table_properties": {"data_size": 31597440, "index_size": 1773120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25926908, "raw_average_key_size": 28, "raw_value_size": 108290870, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925961, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732382921, "oldest_key_time": 0, "file_creation_time": 1732382921, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287286, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:28:44.507211 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287287.sst) /dev/shm/ToplingTemp/Topling-237669-kgm7vC 2024/11/24-01:28:44.594112 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287287.sst): first pass time = 0.09's, 229.145'MB/sec 2024/11/24-01:28:44.774623 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287287.sst): old prealloc_size = 147639500, real_size = 5704944 2024/11/24-01:28:44.774754 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287287.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 138097 prefix = 4 raw-key = 0.0022 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0160 GB zip-val = 0.0054 GB avg-val = 116.13 avg-zval = 39.14 2024/11/24-01:28:44.774902 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287287.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.020 GB 2024/11/24-01:28:44.897445 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00/287287.sst): second pass time = 0.18's, 89.182'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.12's, 51.351'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2406.978'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.334'MB/sec write SST data time = 0.02's, 328.028'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.168'MB, throughput = 6126.034'MB/sec zip my value time = 0.18's, unzip length = 0.016'GB zip my value throughput = 89.182'MB/sec zip pipeline throughput = 89.182'MB/sec entries = 138097 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.13 avg-zval = 39.14 usrkeys = 138097 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.13 avg-zval = 39.14 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 138097 bytes = 466096 } seq expand size = 966679 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0028 GB value = 0.0160 GB dict = 0.17 MB all = 0.0199 GB } __ZipSize{ index = 0.0003 GB value = 0.0054 GB dict = 0.17 MB all = 0.0063 GB } UnZip/Zip{ index = 10.3939 value = 2.9673 dict = 1.00 all = 3.1562 } Zip/UnZip{ index = 0.0962 value = 0.3370 dict = 1.00 all = 0.3168 } ---------------------------- total value len = 10.214165 GB avg = 0.161 KB (by entry num) total key len = 1.749325 GB avg = 0.028 KB total ukey len = 1.218238 GB avg = 0.016 KB total ukey num = 0.076568429 Billion total entry num = 0.063376158 Billion write speed all = 0.200686997 MB/sec (with seq num) write speed all = 0.192181940 MB/sec (without seq num) 2024/11/24-01:28:44.912673 139957265520192 [compaction_job.cc:1948] [default] [JOB 39346] Generated table #287287: 138097 keys, 6306207 bytes, temperature: kWarm 2024/11/24-01:28:44.912730 139957265520192 EVENT_LOG_v1 {"time_micros": 1732382924912702, "cf_name": "default", "job": 39346, "event": "table_file_creation", "file_number": 287287, "file_size": 6306207, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680463, "largest_seqno": 4372192957, "table_properties": {"data_size": 5404512, "index_size": 265728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3866716, "raw_average_key_size": 28, "raw_value_size": 16036890, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 138097, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732382924, "oldest_key_time": 0, "file_creation_time": 1732382924, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287287, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:28:44.923752 139957265520192 [compaction_job.cc:703] job-39346: subcompact[0], size: 0.433719 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39346/att-00 [287275,287276,287277,287278,287279,287280,287281,287282,287283,287284,287285,287286,287287] 2024/11/24-01:28:44.999582 139957265520192 (Original Log Time 2024/11/24-01:28:05.945044) EVENT_LOG_v1 {"time_micros": 1732382885944995, "job": 39346, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [287223], "files_L3": [287256, 287257, 287183, 287184, 287185, 287186, 287187, 287188, 287153, 287154, 287155, 287156, 287157, 287258], "score": 0.719981, "input_data_size": 461678541, "oldest_snapshot_seqno": 4372561610} 2024/11/24-01:28:44.999585 139957265520192 (Original Log Time 2024/11/24-01:28:44.998516) [compaction_job.cc:2022] [default] [JOB 39346] Compacted 1@2 + 14@3 files to L3 => 433719471 bytes 2024/11/24-01:28:44.999587 139957265520192 (Original Log Time 2024/11/24-01:28:44.999516) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 13 0 0 0] max score 0.02, MB/sec: 11.8 rd, 11.1 wr, level 3, files in(1, 14) out(13 +0 blob) MB in(27.2, 413.1 +0.0 blob) out(413.6 +0.0 blob), read-write-amplify(31.4) write-amplify(15.2) OK, records in: 11127450, records dropped: 109456 output_compression: Snappy 2024/11/24-01:28:44.999592 139957265520192 (Original Log Time 2024/11/24-01:28:44.999561) EVENT_LOG_v1 {"time_micros": 1732382924999538, "cf": "default", "job": 39346, "event": "compaction_finished", "compaction_time_micros": 38978714, "compaction_time_cpu_micros": 16682350, "output_level": 3, "num_output_files": 13, "total_output_size": 433719471, "num_input_records": 11127450, "num_output_records": 11017994, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 301523800, "file_range_sync_nanos": 0, "file_fsync_nanos": 9129, "file_prepare_write_nanos": 34563, "lsm_state": [0, 0, 0, 13, 0, 0, 0]} 2024/11/24-01:28:45.135849 139957265520192 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732382885,"executesMs":39187,"compactionJobId":39346,"attempt":0,"compactionInputRawBytes":1612466196,"compactionInputZipBytes":461678541,"compactionOutputRawBytes":0,"compactionOutputZipBytes":433719471,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-c3aede947d404205316665ada375603d-e5b6c8b408b37e9b-00"}}, response = 2024/11/24-01:28:45.135925 139957265520192 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-39346/att-00: olev 3, work 39.187 s, result 7.649 ms, install 1.091 ms, input{raw 1.502 GiB zip 440.291 MiB}