2024/11/22-19:53:30.634186 140024928073280 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00: fork to child time = 0.020361 sec 2024/11/22-19:53:30.635034 140024928073280 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00 2024/11/22-19:53:30.642109 140024928073280 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00 2024/11/22-19:53:30.643048 140024928073280 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-19:53:30.643348 140024928073280 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-19:53:30.688455 140024928073280 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-09418/att-00/MANIFEST-206347 2024/11/22-19:53:30.728947 140024928073280 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-09418/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 224468, last_sequence is 3518923582, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 224276 2024/11/22-19:53:30.728955 140024928073280 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-19:53:30.729042 140024928073280 [version_set.cc:5721] Creating manifest 224468 2024/11/22-19:53:30.762886 140024928073280 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-19:53:30.783031 140024928073280 [compaction_job.cc:2437] [default] [JOB 9418] Compacting 1@2 + 14@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/22-19:53:30.783049 140024928073280 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 7119 Base level 2, inputs: [224410(27MB)], [224465(35MB) 224466(3766KB) 224379(37MB) 224380(35MB) 224381(11MB) 224346(35MB) 224347(36MB) 224348(35MB) 224349(34MB) 224350(35MB) 224351(35MB) 224352(35MB) 224353(35MB) 224420(22MB)] 2024/11/22-19:53:30.791555 140024928073280 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-19:53:30.791580 140024928073280 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-19:53:30.791595 140024928073280 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-19:53:30.791600 140024928073280 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-19:53:30.791609 140024928073280 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-19:53:31.234440 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224469.sst) /dev/shm/ToplingTemp/Topling-267335-MpkrAP 2024/11/22-19:53:31.815351 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224469.sst): first pass time = 0.58's, 231.066'MB/sec 2024/11/22-19:53:33.790395 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224469.sst): old prealloc_size = 147639500, real_size = 32876272 2024/11/22-19:53:33.790970 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224469.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925588 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 117.01 avg-zval = 33.37 2024/11/22-19:53:33.791769 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224469.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:53:34.707109 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224469.sst): second pass time = 1.97's, 54.860'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.330'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3173.792'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.293'MB/sec write SST data time = 0.13's, 291.427'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33116.198'MB/sec zip my value time = 1.97's, unzip length = 0.108'GB zip my value throughput = 54.860'MB/sec zip pipeline throughput = 98.409'MB/sec entries = 925588 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.01 avg-zval = 33.37 usrkeys = 925588 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.01 avg-zval = 33.37 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925588 bytes = 3239568 } seq expand size = 6479116 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5127 value = 3.5067 dict = 1.00 all = 3.6303 } Zip/UnZip{ index = 0.0951 value = 0.2852 dict = 1.00 all = 0.2755 } ---------------------------- total value len = 0.114780 GB avg = 0.124 KB (by entry num) total key len = 0.025916 GB avg = 0.028 KB total ukey len = 0.018512 GB avg = 0.020 KB total ukey num = 0.000925588 Billion total entry num = 0.000925588 Billion write speed all = 0.003546539 MB/sec (with seq num) write speed all = 0.003359889 MB/sec (without seq num) 2024/11/22-19:53:34.742135 140024928073280 [compaction_job.cc:1948] [default] [JOB 9418] Generated table #224469: 925588 keys, 36971034 bytes, temperature: kWarm 2024/11/22-19:53:34.742224 140024928073280 EVENT_LOG_v1 {"time_micros": 1732276414742169, "cf_name": "default", "job": 9418, "event": "table_file_creation", "file_number": 224469, "file_size": 36971034, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3342305137, "largest_seqno": 3518443171, "table_properties": {"data_size": 30883808, "index_size": 1760896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25916464, "raw_average_key_size": 28, "raw_value_size": 108301324, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925588, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732276411, "oldest_key_time": 0, "file_creation_time": 1732276411, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 224469, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:53:34.779305 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224470.sst) /dev/shm/ToplingTemp/Topling-267335-D3fMBM 2024/11/22-19:53:35.653389 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224470.sst): first pass time = 0.87's, 153.560'MB/sec 2024/11/22-19:53:37.296154 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224470.sst): old prealloc_size = 147639500, real_size = 34615248 2024/11/22-19:53:37.296713 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224470.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928409 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0326 GB avg-val = 116.57 avg-zval = 35.12 2024/11/22-19:53:37.297680 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224470.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:53:38.207176 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224470.sst): second pass time = 1.64's, 65.902'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 42.500'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3309.258'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.297'MB/sec write SST data time = 0.13's, 297.497'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 21130.840'MB/sec zip my value time = 1.64's, unzip length = 0.108'GB zip my value throughput = 65.902'MB/sec zip pipeline throughput = 130.634'MB/sec entries = 928409 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.57 avg-zval = 35.12 usrkeys = 928409 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.57 avg-zval = 35.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928409 bytes = 3249440 } seq expand size = 6498863 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0326 GB dict = 1.08 MB all = 0.0387 GB } UnZip/Zip{ index = 10.4453 value = 3.3192 dict = 1.00 all = 3.4665 } Zip/UnZip{ index = 0.0957 value = 0.3013 dict = 1.00 all = 0.2885 } ---------------------------- total value len = 0.229502 GB avg = 0.124 KB (by entry num) total key len = 0.051912 GB avg = 0.028 KB total ukey len = 0.037080 GB avg = 0.020 KB total ukey num = 0.001853997 Billion total entry num = 0.001853997 Billion write speed all = 0.007092949 MB/sec (with seq num) write speed all = 0.006719113 MB/sec (without seq num) 2024/11/22-19:53:38.260209 140024928073280 [compaction_job.cc:1948] [default] [JOB 9418] Generated table #224470: 928409 keys, 38718054 bytes, temperature: kWarm 2024/11/22-19:53:38.260260 140024928073280 EVENT_LOG_v1 {"time_micros": 1732276418260232, "cf_name": "default", "job": 9418, "event": "table_file_creation", "file_number": 224470, "file_size": 38718054, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701165, "largest_seqno": 3518443193, "table_properties": {"data_size": 32605312, "index_size": 1777664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25995452, "raw_average_key_size": 28, "raw_value_size": 108222306, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928409, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732276414, "oldest_key_time": 0, "file_creation_time": 1732276414, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 224470, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:53:38.304406 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224471.sst) /dev/shm/ToplingTemp/Topling-267335-XNWBBP 2024/11/22-19:53:39.123109 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224471.sst): first pass time = 0.82's, 163.947'MB/sec 2024/11/22-19:53:40.722538 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224471.sst): old prealloc_size = 147639500, real_size = 32599344 2024/11/22-19:53:40.723125 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224471.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923538 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0306 GB avg-val = 117.33 avg-zval = 33.15 2024/11/22-19:53:40.723868 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224471.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:53:41.611434 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224471.sst): second pass time = 1.60's, 67.801'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 41.146'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3108.198'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.305'MB/sec write SST data time = 0.13's, 280.371'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 26960.038'MB/sec zip my value time = 1.60's, unzip length = 0.108'GB zip my value throughput = 67.801'MB/sec zip pipeline throughput = 124.214'MB/sec entries = 923538 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.33 avg-zval = 33.15 usrkeys = 923538 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.33 avg-zval = 33.15 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923538 bytes = 3116960 } seq expand size = 6464766 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5139 value = 3.5398 dict = 1.00 all = 3.6698 } Zip/UnZip{ index = 0.0951 value = 0.2825 dict = 1.00 all = 0.2725 } ---------------------------- total value len = 0.344325 GB avg = 0.124 KB (by entry num) total key len = 0.077771 GB avg = 0.028 KB total ukey len = 0.055551 GB avg = 0.020 KB total ukey num = 0.002777535 Billion total entry num = 0.002777535 Billion write speed all = 0.010637902 MB/sec (with seq num) write speed all = 0.010077894 MB/sec (without seq num) 2024/11/22-19:53:41.664889 140024928073280 [compaction_job.cc:1948] [default] [JOB 9418] Generated table #224471: 923538 keys, 36574076 bytes, temperature: kWarm 2024/11/22-19:53:41.664942 140024928073280 EVENT_LOG_v1 {"time_micros": 1732276421664913, "cf_name": "default", "job": 9418, "event": "table_file_creation", "file_number": 224471, "file_size": 36574076, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3388810268, "largest_seqno": 3518443206, "table_properties": {"data_size": 30611488, "index_size": 1756800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25859064, "raw_average_key_size": 28, "raw_value_size": 108358788, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923538, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732276418, "oldest_key_time": 0, "file_creation_time": 1732276418, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 224471, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:53:41.710185 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224472.sst) /dev/shm/ToplingTemp/Topling-267335-XYBNxQ 2024/11/22-19:53:42.621583 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224472.sst): first pass time = 0.91's, 147.272'MB/sec 2024/11/22-19:53:44.159158 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224472.sst): old prealloc_size = 147639500, real_size = 33293888 2024/11/22-19:53:44.159692 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224472.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923528 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0313 GB avg-val = 117.33 avg-zval = 33.90 2024/11/22-19:53:44.160466 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224472.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:53:45.079314 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224472.sst): second pass time = 1.54's, 70.505'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.628'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3413.617'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.292'MB/sec write SST data time = 0.13's, 288.223'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 26880.010'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.505'MB/sec zip pipeline throughput = 122.878'MB/sec entries = 923528 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.33 avg-zval = 33.90 usrkeys = 923528 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.33 avg-zval = 33.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923528 bytes = 3232368 } seq expand size = 6464696 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5111 value = 3.4613 dict = 1.00 all = 3.5903 } Zip/UnZip{ index = 0.0951 value = 0.2889 dict = 1.00 all = 0.2785 } ---------------------------- total value len = 0.459149 GB avg = 0.124 KB (by entry num) total key len = 0.103630 GB avg = 0.028 KB total ukey len = 0.074021 GB avg = 0.020 KB total ukey num = 0.003701063 Billion total entry num = 0.003701063 Billion write speed all = 0.014182219 MB/sec (with seq num) write speed all = 0.013436074 MB/sec (without seq num) 2024/11/22-19:53:45.113721 140024928073280 [compaction_job.cc:1948] [default] [JOB 9418] Generated table #224472: 923528 keys, 37383658 bytes, temperature: kWarm 2024/11/22-19:53:45.113775 140024928073280 EVENT_LOG_v1 {"time_micros": 1732276425113746, "cf_name": "default", "job": 9418, "event": "table_file_creation", "file_number": 224472, "file_size": 37383658, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 3518443216, "table_properties": {"data_size": 31305584, "index_size": 1757248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25858784, "raw_average_key_size": 28, "raw_value_size": 108358964, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923528, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732276421, "oldest_key_time": 0, "file_creation_time": 1732276421, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 224472, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:53:45.150152 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224473.sst) /dev/shm/ToplingTemp/Topling-267335-ktEIWO 2024/11/22-19:53:45.949799 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224473.sst): first pass time = 0.80's, 167.854'MB/sec 2024/11/22-19:53:47.556205 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224473.sst): old prealloc_size = 147639500, real_size = 33336368 2024/11/22-19:53:47.556741 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224473.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925843 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.97 avg-zval = 33.85 2024/11/22-19:53:47.557385 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224473.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:53:48.427626 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224473.sst): second pass time = 1.61's, 67.470'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.050'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3435.271'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.312'MB/sec write SST data time = 0.13's, 282.707'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 36033.811'MB/sec zip my value time = 1.61's, unzip length = 0.108'GB zip my value throughput = 67.470'MB/sec zip pipeline throughput = 104.976'MB/sec entries = 925843 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.97 avg-zval = 33.85 usrkeys = 925843 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.13 avg-zval = 33.85 TagRS{ kinds = RS_Key0_TagN bytes = 144760 } TagArray{ size = 682422 bytes = 2303184 } seq expand size = 4776954 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5114 value = 3.4551 dict = 1.00 all = 3.6628 } Zip/UnZip{ index = 0.0951 value = 0.2894 dict = 1.00 all = 0.2730 } ---------------------------- total value len = 0.572220 GB avg = 0.124 KB (by entry num) total key len = 0.129553 GB avg = 0.028 KB total ukey len = 0.092538 GB avg = 0.020 KB total ukey num = 0.004626906 Billion total entry num = 0.004626906 Billion write speed all = 0.017683442 MB/sec (with seq num) write speed all = 0.016750724 MB/sec (without seq num) 2024/11/22-19:53:48.459948 140024928073280 [compaction_job.cc:1948] [default] [JOB 9418] Generated table #224473: 925843 keys, 36643102 bytes, temperature: kWarm 2024/11/22-19:53:48.460009 140024928073280 EVENT_LOG_v1 {"time_micros": 1732276428459977, "cf_name": "default", "job": 9418, "event": "table_file_creation", "file_number": 224473, "file_size": 36643102, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3518443226, "table_properties": {"data_size": 31343136, "index_size": 1761600, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25923604, "raw_average_key_size": 28, "raw_value_size": 108294126, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925843, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732276425, "oldest_key_time": 0, "file_creation_time": 1732276425, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 224473, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:53:48.497316 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224474.sst) /dev/shm/ToplingTemp/Topling-267335-hIN2LO 2024/11/22-19:53:49.311977 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224474.sst): first pass time = 0.81's, 164.761'MB/sec 2024/11/22-19:53:50.887400 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224474.sst): old prealloc_size = 147639500, real_size = 35481120 2024/11/22-19:53:50.887954 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224474.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923026 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0335 GB avg-val = 117.41 avg-zval = 36.28 2024/11/22-19:53:50.888434 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224474.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:53:51.691564 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224474.sst): second pass time = 1.57's, 68.819'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.80's, 47.428'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3328.058'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.341'MB/sec write SST data time = 0.13's, 298.621'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32677.048'MB/sec zip my value time = 1.57's, unzip length = 0.108'GB zip my value throughput = 68.819'MB/sec zip pipeline throughput = 76.339'MB/sec entries = 923026 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.41 avg-zval = 36.28 usrkeys = 923026 avg-key = 20.00 avg-zkey = 1.91 avg-val = 120.99 avg-zval = 36.28 TagRS{ kinds = RS_Key0_TagN bytes = 144320 } TagArray{ size = 472148 bytes = 1652528 } seq expand size = 3305036 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0335 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.4460 value = 3.2367 dict = 1.00 all = 3.5191 } Zip/UnZip{ index = 0.0957 value = 0.3090 dict = 1.00 all = 0.2842 } ---------------------------- total value len = 0.683898 GB avg = 0.123 KB (by entry num) total key len = 0.155398 GB avg = 0.028 KB total ukey len = 0.110999 GB avg = 0.020 KB total ukey num = 0.005549932 Billion total entry num = 0.005549932 Billion write speed all = 0.021147033 MB/sec (with seq num) write speed all = 0.020028337 MB/sec (without seq num) 2024/11/22-19:53:51.725158 140024928073280 [compaction_job.cc:1948] [default] [JOB 9418] Generated table #224474: 923026 keys, 38139537 bytes, temperature: kWarm 2024/11/22-19:53:51.725219 140024928073280 EVENT_LOG_v1 {"time_micros": 1732276431725187, "cf_name": "default", "job": 9418, "event": "table_file_creation", "file_number": 224474, "file_size": 38139537, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3484837085, "table_properties": {"data_size": 33482960, "index_size": 1767232, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25844728, "raw_average_key_size": 28, "raw_value_size": 108373004, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923026, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732276428, "oldest_key_time": 0, "file_creation_time": 1732276428, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 224474, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:53:51.770776 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224475.sst) /dev/shm/ToplingTemp/Topling-267335-oZo6PM 2024/11/22-19:53:52.604378 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224475.sst): first pass time = 0.83's, 161.019'MB/sec 2024/11/22-19:53:54.154121 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224475.sst): old prealloc_size = 147639500, real_size = 35245712 2024/11/22-19:53:54.154642 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224475.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926000 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0333 GB avg-val = 116.94 avg-zval = 35.91 2024/11/22-19:53:54.155438 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224475.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:53:55.045857 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224475.sst): second pass time = 1.55's, 69.922'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 44.118'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3535.549'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.303'MB/sec write SST data time = 0.13's, 307.632'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32743.977'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 69.922'MB/sec zip pipeline throughput = 77.953'MB/sec entries = 926000 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.94 avg-zval = 35.91 usrkeys = 926000 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.94 avg-zval = 35.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926000 bytes = 3241008 } seq expand size = 6482000 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0333 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.5116 value = 3.2566 dict = 1.00 all = 3.4116 } Zip/UnZip{ index = 0.0951 value = 0.3071 dict = 1.00 all = 0.2931 } ---------------------------- total value len = 0.798670 GB avg = 0.123 KB (by entry num) total key len = 0.181326 GB avg = 0.028 KB total ukey len = 0.129519 GB avg = 0.020 KB total ukey num = 0.006475932 Billion total entry num = 0.006475932 Billion write speed all = 0.024690041 MB/sec (with seq num) write speed all = 0.023384803 MB/sec (without seq num) 2024/11/22-19:53:55.082857 140024928073280 [compaction_job.cc:1948] [default] [JOB 9418] Generated table #224475: 926000 keys, 39341592 bytes, temperature: kWarm 2024/11/22-19:53:55.082919 140024928073280 EVENT_LOG_v1 {"time_micros": 1732276435082887, "cf_name": "default", "job": 9418, "event": "table_file_creation", "file_number": 224475, "file_size": 39341592, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701550, "largest_seqno": 3518442679, "table_properties": {"data_size": 33252160, "index_size": 1761856, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25928000, "raw_average_key_size": 28, "raw_value_size": 108289844, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926000, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732276431, "oldest_key_time": 0, "file_creation_time": 1732276431, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 224475, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:53:55.140858 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224476.sst) /dev/shm/ToplingTemp/Topling-267335-83CUjP 2024/11/22-19:53:55.958891 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224476.sst): first pass time = 0.82's, 164.082'MB/sec 2024/11/22-19:53:57.498204 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224476.sst): old prealloc_size = 147639500, real_size = 31621728 2024/11/22-19:53:57.498718 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224476.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923479 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0296 GB avg-val = 117.34 avg-zval = 32.09 2024/11/22-19:53:57.499518 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224476.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:53:58.352531 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224476.sst): second pass time = 1.54's, 70.431'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 41.801'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3571.677'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 280.970'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 31916.460'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.431'MB/sec zip pipeline throughput = 111.573'MB/sec entries = 923479 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.34 avg-zval = 32.09 usrkeys = 923479 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.34 avg-zval = 32.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923479 bytes = 3232192 } seq expand size = 6464353 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5143 value = 3.6566 dict = 1.00 all = 3.7584 } Zip/UnZip{ index = 0.0951 value = 0.2735 dict = 1.00 all = 0.2661 } ---------------------------- total value len = 0.913494 GB avg = 0.123 KB (by entry num) total key len = 0.207184 GB avg = 0.028 KB total ukey len = 0.147988 GB avg = 0.020 KB total ukey num = 0.007399411 Billion total entry num = 0.007399411 Billion write speed all = 0.028232038 MB/sec (with seq num) write speed all = 0.026740794 MB/sec (without seq num) 2024/11/22-19:53:58.385899 140024928073280 [compaction_job.cc:1948] [default] [JOB 9418] Generated table #224476: 923479 keys, 35711614 bytes, temperature: kWarm 2024/11/22-19:53:58.385962 140024928073280 EVENT_LOG_v1 {"time_micros": 1732276438385930, "cf_name": "default", "job": 9418, "event": "table_file_creation", "file_number": 224476, "file_size": 35711614, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3344892904, "largest_seqno": 3518442830, "table_properties": {"data_size": 29634064, "index_size": 1756608, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25857412, "raw_average_key_size": 28, "raw_value_size": 108360334, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923479, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732276435, "oldest_key_time": 0, "file_creation_time": 1732276435, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 224476, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:53:58.420560 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224477.sst) /dev/shm/ToplingTemp/Topling-267335-GulrbR 2024/11/22-19:53:59.267603 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224477.sst): first pass time = 0.85's, 158.464'MB/sec 2024/11/22-19:54:00.862019 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224477.sst): old prealloc_size = 147639500, real_size = 33061664 2024/11/22-19:54:00.862535 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224477.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923425 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0311 GB avg-val = 117.35 avg-zval = 33.65 2024/11/22-19:54:00.863365 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224477.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:54:01.775618 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224477.sst): second pass time = 1.59's, 67.997'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 40.538'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3546.892'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.294'MB/sec write SST data time = 0.13's, 286.974'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33074.475'MB/sec zip my value time = 1.59's, unzip length = 0.108'GB zip my value throughput = 67.997'MB/sec zip pipeline throughput = 131.827'MB/sec entries = 923425 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.35 avg-zval = 33.65 usrkeys = 923425 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.35 avg-zval = 33.65 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923425 bytes = 3116576 } seq expand size = 6463975 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5114 value = 3.4873 dict = 1.00 all = 3.6240 } Zip/UnZip{ index = 0.0951 value = 0.2868 dict = 1.00 all = 0.2759 } ---------------------------- total value len = 1.028320 GB avg = 0.124 KB (by entry num) total key len = 0.233039 GB avg = 0.028 KB total ukey len = 0.166457 GB avg = 0.020 KB total ukey num = 0.008322836 Billion total entry num = 0.008322836 Billion write speed all = 0.031773342 MB/sec (with seq num) write speed all = 0.030096140 MB/sec (without seq num) 2024/11/22-19:54:01.817749 140024928073280 [compaction_job.cc:1948] [default] [JOB 9418] Generated table #224477: 923425 keys, 37035894 bytes, temperature: kWarm 2024/11/22-19:54:01.818730 140024928073280 EVENT_LOG_v1 {"time_micros": 1732276441818689, "cf_name": "default", "job": 9418, "event": "table_file_creation", "file_number": 224477, "file_size": 37035894, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268533, "largest_seqno": 3518442702, "table_properties": {"data_size": 31073616, "index_size": 1756992, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25855900, "raw_average_key_size": 28, "raw_value_size": 108361854, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923425, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732276438, "oldest_key_time": 0, "file_creation_time": 1732276438, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 224477, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:54:01.860033 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224478.sst) /dev/shm/ToplingTemp/Topling-267335-YehEHQ 2024/11/22-19:54:02.737471 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224478.sst): first pass time = 0.88's, 152.972'MB/sec 2024/11/22-19:54:04.352125 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224478.sst): old prealloc_size = 147639500, real_size = 33760896 2024/11/22-19:54:04.352653 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224478.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923475 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0318 GB avg-val = 117.34 avg-zval = 34.39 2024/11/22-19:54:04.353427 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224478.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:54:05.290343 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224478.sst): second pass time = 1.61's, 67.141'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 40.221'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3495.938'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.286'MB/sec write SST data time = 0.13's, 291.463'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 30066.585'MB/sec zip my value time = 1.61's, unzip length = 0.108'GB zip my value throughput = 67.141'MB/sec zip pipeline throughput = 133.021'MB/sec entries = 923475 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.34 avg-zval = 34.39 usrkeys = 923475 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.34 avg-zval = 34.39 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923475 bytes = 3116736 } seq expand size = 6464325 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.4450 value = 3.4117 dict = 1.00 all = 3.5568 } Zip/UnZip{ index = 0.0957 value = 0.2931 dict = 1.00 all = 0.2812 } ---------------------------- total value len = 1.143145 GB avg = 0.124 KB (by entry num) total key len = 0.258897 GB avg = 0.028 KB total ukey len = 0.184926 GB avg = 0.020 KB total ukey num = 0.009246311 Billion total entry num = 0.009246311 Billion write speed all = 0.035313963 MB/sec (with seq num) write speed all = 0.033450829 MB/sec (without seq num) 2024/11/22-19:54:05.323490 140024928073280 [compaction_job.cc:1948] [default] [JOB 9418] Generated table #224478: 923475 keys, 37735378 bytes, temperature: kWarm 2024/11/22-19:54:05.323546 140024928073280 EVENT_LOG_v1 {"time_micros": 1732276445323516, "cf_name": "default", "job": 9418, "event": "table_file_creation", "file_number": 224478, "file_size": 37735378, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268543, "largest_seqno": 3518442721, "table_properties": {"data_size": 31761584, "index_size": 1768256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25857300, "raw_average_key_size": 28, "raw_value_size": 108360466, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923475, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732276441, "oldest_key_time": 0, "file_creation_time": 1732276441, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 224478, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:54:05.359403 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224479.sst) /dev/shm/ToplingTemp/Topling-267335-soRwlQ 2024/11/22-19:54:06.278059 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224479.sst): first pass time = 0.92's, 146.109'MB/sec 2024/11/22-19:54:07.874112 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224479.sst): old prealloc_size = 147639500, real_size = 33483200 2024/11/22-19:54:07.874644 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224479.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923586 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0315 GB avg-val = 117.32 avg-zval = 34.09 2024/11/22-19:54:07.875439 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224479.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:54:08.813513 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224479.sst): second pass time = 1.59's, 67.940'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.875'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3463.222'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.285'MB/sec write SST data time = 0.13's, 291.731'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 30031.454'MB/sec zip my value time = 1.59's, unzip length = 0.108'GB zip my value throughput = 67.940'MB/sec zip pipeline throughput = 128.106'MB/sec entries = 923586 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.32 avg-zval = 34.09 usrkeys = 923586 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.32 avg-zval = 34.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923586 bytes = 3117120 } seq expand size = 6465102 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4433 value = 3.4417 dict = 1.00 all = 3.5831 } Zip/UnZip{ index = 0.0958 value = 0.2906 dict = 1.00 all = 0.2791 } ---------------------------- total value len = 1.257968 GB avg = 0.124 KB (by entry num) total key len = 0.284757 GB avg = 0.028 KB total ukey len = 0.203398 GB avg = 0.020 KB total ukey num = 0.010169897 Billion total entry num = 0.010169897 Billion write speed all = 0.038853967 MB/sec (with seq num) write speed all = 0.036804913 MB/sec (without seq num) 2024/11/22-19:54:08.847242 140024928073280 [compaction_job.cc:1948] [default] [JOB 9418] Generated table #224479: 923586 keys, 37458100 bytes, temperature: kWarm 2024/11/22-19:54:08.847304 140024928073280 EVENT_LOG_v1 {"time_micros": 1732276448847273, "cf_name": "default", "job": 9418, "event": "table_file_creation", "file_number": 224479, "file_size": 37458100, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268562, "largest_seqno": 3518442751, "table_properties": {"data_size": 31483376, "index_size": 1768768, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25860408, "raw_average_key_size": 28, "raw_value_size": 108357340, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923586, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732276445, "oldest_key_time": 0, "file_creation_time": 1732276445, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 224479, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:54:08.884983 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224480.sst) /dev/shm/ToplingTemp/Topling-267335-bD15UO 2024/11/22-19:54:09.645358 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224480.sst): first pass time = 0.76's, 176.525'MB/sec 2024/11/22-19:54:11.160682 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224480.sst): old prealloc_size = 147639500, real_size = 34167024 2024/11/22-19:54:11.161241 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224480.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927625 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0322 GB avg-val = 116.69 avg-zval = 34.68 2024/11/22-19:54:11.162012 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224480.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:54:12.074976 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224480.sst): second pass time = 1.51's, 71.466'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 41.729'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3284.525'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.296'MB/sec write SST data time = 0.13's, 294.243'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 29899.607'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.466'MB/sec zip pipeline throughput = 113.930'MB/sec entries = 927625 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.69 avg-zval = 34.68 usrkeys = 927625 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.69 avg-zval = 34.68 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927625 bytes = 3130752 } seq expand size = 6493375 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0322 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.5114 value = 3.3648 dict = 1.00 all = 3.5180 } Zip/UnZip{ index = 0.0951 value = 0.2972 dict = 1.00 all = 0.2843 } ---------------------------- total value len = 1.372705 GB avg = 0.124 KB (by entry num) total key len = 0.310731 GB avg = 0.028 KB total ukey len = 0.221950 GB avg = 0.020 KB total ukey num = 0.011097522 Billion total entry num = 0.011097522 Billion write speed all = 0.042394337 MB/sec (with seq num) write speed all = 0.040158566 MB/sec (without seq num) 2024/11/22-19:54:12.109333 140024928073280 [compaction_job.cc:1948] [default] [JOB 9418] Generated table #224480: 927625 keys, 38151674 bytes, temperature: kWarm 2024/11/22-19:54:12.109395 140024928073280 EVENT_LOG_v1 {"time_micros": 1732276452109362, "cf_name": "default", "job": 9418, "event": "table_file_creation", "file_number": 224480, "file_size": 38151674, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 3518442788, "table_properties": {"data_size": 32169952, "index_size": 1764992, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25973500, "raw_average_key_size": 28, "raw_value_size": 108244310, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927625, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732276448, "oldest_key_time": 0, "file_creation_time": 1732276448, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 224480, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:54:12.157376 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224481.sst) /dev/shm/ToplingTemp/Topling-267335-qcuJkQ 2024/11/22-19:54:12.251035 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224481.sst): first pass time = 0.09's, 234.406'MB/sec 2024/11/22-19:54:12.475768 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224481.sst): old prealloc_size = 147639500, real_size = 5918992 2024/11/22-19:54:12.475915 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224481.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 154321 prefix = 4 raw-key = 0.0025 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0176 GB zip-val = 0.0056 GB avg-val = 114.20 avg-zval = 36.18 2024/11/22-19:54:12.476061 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224481.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.022 GB 2024/11/22-19:54:12.619112 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00/224481.sst): second pass time = 0.22's, 78.603'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.14's, 45.699'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2540.895'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.319'MB/sec write SST data time = 0.02's, 296.475'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.185'MB, throughput = 6809.247'MB/sec zip my value time = 0.22's, unzip length = 0.018'GB zip my value throughput = 78.603'MB/sec zip pipeline throughput = 150.267'MB/sec entries = 154321 avg-key = 28.00 avg-zkey = 1.92 avg-val = 114.20 avg-zval = 36.18 usrkeys = 154321 avg-key = 20.00 avg-zkey = 1.92 avg-val = 120.85 avg-zval = 36.18 TagRS{ kinds = RS_Key0_TagN bytes = 24200 } TagArray{ size = 146512 bytes = 457872 } seq expand size = 1025584 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0031 GB value = 0.0176 GB dict = 0.18 MB all = 0.0219 GB } __ZipSize{ index = 0.0003 GB value = 0.0056 GB dict = 0.18 MB all = 0.0066 GB } UnZip/Zip{ index = 10.3956 value = 3.1565 dict = 1.00 all = 3.3503 } Zip/UnZip{ index = 0.0962 value = 0.3168 dict = 1.00 all = 0.2985 } ---------------------------- total value len = 1.391355 GB avg = 0.124 KB (by entry num) total key len = 0.315052 GB avg = 0.028 KB total ukey len = 0.225037 GB avg = 0.020 KB total ukey num = 0.011251843 Billion total entry num = 0.011251843 Billion write speed all = 0.042972217 MB/sec (with seq num) write speed all = 0.040705387 MB/sec (without seq num) 2024/11/22-19:54:12.629750 140024928073280 [compaction_job.cc:1948] [default] [JOB 9418] Generated table #224481: 154321 keys, 6550051 bytes, temperature: kWarm 2024/11/22-19:54:12.629822 140024928073280 EVENT_LOG_v1 {"time_micros": 1732276452629775, "cf_name": "default", "job": 9418, "event": "table_file_creation", "file_number": 224481, "file_size": 6550051, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3518443083, "table_properties": {"data_size": 5583360, "index_size": 296896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4320988, "raw_average_key_size": 28, "raw_value_size": 17623830, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 154321, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732276452, "oldest_key_time": 0, "file_creation_time": 1732276452, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 224481, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:54:12.640003 140024928073280 [compaction_job.cc:703] job-09418: subcompact[0], size: 0.456414 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09418/att-00 [224469,224470,224471,224472,224473,224474,224475,224476,224477,224478,224479,224480,224481] 2024/11/22-19:54:12.720082 140024928073280 (Original Log Time 2024/11/22-19:53:30.783090) EVENT_LOG_v1 {"time_micros": 1732276410783061, "job": 9418, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [224410], "files_L3": [224465, 224466, 224379, 224380, 224381, 224346, 224347, 224348, 224349, 224350, 224351, 224352, 224353, 224420], "score": 0.988817, "input_data_size": 478860584, "oldest_snapshot_seqno": 3518923534} 2024/11/22-19:54:12.720085 140024928073280 (Original Log Time 2024/11/22-19:54:12.719285) [compaction_job.cc:2022] [default] [JOB 9418] Compacted 1@2 + 14@3 files to L3 => 456413764 bytes 2024/11/22-19:54:12.720086 140024928073280 (Original Log Time 2024/11/22-19:54:12.720016) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 13 0 0 0] max score 0.02, MB/sec: 11.4 rd, 10.9 wr, level 3, files in(1, 14) out(13 +0 blob) MB in(27.1, 429.5 +0.0 blob) out(435.3 +0.0 blob), read-write-amplify(32.9) write-amplify(16.0) OK, records in: 11331385, records dropped: 79542 output_compression: Snappy 2024/11/22-19:54:12.720092 140024928073280 (Original Log Time 2024/11/22-19:54:12.720061) EVENT_LOG_v1 {"time_micros": 1732276452720039, "cf": "default", "job": 9418, "event": "compaction_finished", "compaction_time_micros": 41856920, "compaction_time_cpu_micros": 17756317, "output_level": 3, "num_output_files": 13, "total_output_size": 456413764, "num_input_records": 11331385, "num_output_records": 11251843, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 289442798, "file_range_sync_nanos": 0, "file_fsync_nanos": 11423, "file_prepare_write_nanos": 30330, "lsm_state": [0, 0, 0, 13, 0, 0, 0]} 2024/11/22-19:54:12.854742 140024928073280 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732276410,"executesMs":42079,"compactionJobId":9418,"attempt":0,"compactionInputRawBytes":1643853058,"compactionInputZipBytes":478860584,"compactionOutputRawBytes":0,"compactionOutputZipBytes":456413764,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-e69f21bb364b3f106a3afe6a63977981-778996279572e07f-00"}}, response = 2024/11/22-19:54:12.854831 140024928073280 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-09418/att-00: olev 3, work 42.080 s, result 5.207 ms, install 0.825 ms, input{raw 1.531 GiB zip 456.677 MiB}