2024/11/23-21:00:58.257582 139957668369984 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00: fork to child time = 0.030138 sec 2024/11/23-21:00:58.258559 139957668369984 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00 2024/11/23-21:00:58.266920 139957668369984 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00 2024/11/23-21:00:58.268123 139957668369984 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-21:00:58.268465 139957668369984 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-21:00:58.318208 139957668369984 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-35738/att-00/MANIFEST-206347 2024/11/23-21:00:58.354160 139957668369984 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-35738/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 280507, last_sequence is 4259898529, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 280241 2024/11/23-21:00:58.354177 139957668369984 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-21:00:58.354476 139957668369984 [version_set.cc:5721] Creating manifest 280507 2024/11/23-21:00:58.391463 139957668369984 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-21:00:58.414358 139957668369984 [compaction_job.cc:2437] [default] [JOB 35738] Compacting 1@2 + 22@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-21:00:58.414395 139957668369984 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 26752 Base level 2, inputs: [280499(23MB)], [280504(38MB) 280505(3486KB) 280412(38MB) 280413(38MB) 280414(35MB) 280415(34MB) 280416(37MB) 280417(33MB) 280418(37MB) 280419(34MB) 280420(37MB) 280421(37MB) 280422(36MB) 280492(36MB) 280493(36MB) 280494(35MB) 280495(35MB) 280496(35MB) 280497(35MB) 280498(28MB) 280394(36MB) 280395(27MB)] 2024/11/23-21:00:58.422534 139957668369984 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-21:00:58.422578 139957668369984 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-21:00:58.422608 139957668369984 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-21:00:58.422618 139957668369984 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-21:00:58.422632 139957668369984 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-21:00:59.162329 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280508.sst) /dev/shm/ToplingTemp/Topling-188197-3A7NWK 2024/11/23-21:00:59.826118 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280508.sst): first pass time = 0.66's, 202.211'MB/sec 2024/11/23-21:01:01.370558 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280508.sst): old prealloc_size = 147639500, real_size = 36195504 2024/11/23-21:01:01.371319 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280508.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931071 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0342 GB avg-val = 116.15 avg-zval = 36.72 2024/11/23-21:01:01.372342 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280508.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:01:02.436216 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280508.sst): second pass time = 1.54's, 70.077'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.07's, 38.045'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2408.487'MB/sec (index lex order gen) rebuild zvType time = 0.91's, 0.256'MB/sec write SST data time = 0.16's, 257.809'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33539.782'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.077'MB/sec zip pipeline throughput = 70.077'MB/sec entries = 931071 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.15 avg-zval = 36.72 usrkeys = 931071 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.15 avg-zval = 36.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931071 bytes = 3491536 } seq expand size = 6517497 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0342 GB dict = 1.08 MB all = 0.0405 GB } UnZip/Zip{ index = 10.5131 value = 3.1630 dict = 1.00 all = 3.3105 } Zip/UnZip{ index = 0.0951 value = 0.3162 dict = 1.00 all = 0.3021 } ---------------------------- total value len = 8.963285 GB avg = 0.168 KB (by entry num) total key len = 1.466891 GB avg = 0.028 KB total ukey len = 1.016499 GB avg = 0.015 KB total ukey num = 0.066481506 Billion total entry num = 0.053289235 Billion write speed all = 0.239497660 MB/sec (with seq num) write speed all = 0.229708643 MB/sec (without seq num) 2024/11/23-21:01:02.486916 139957668369984 [compaction_job.cc:1948] [default] [JOB 35738] Generated table #280508: 931071 keys, 40542628 bytes, temperature: kWarm 2024/11/23-21:01:02.487031 139957668369984 EVENT_LOG_v1 {"time_micros": 1732366862486953, "cf_name": "default", "job": 35738, "event": "table_file_creation", "file_number": 280508, "file_size": 40542628, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3591110068, "largest_seqno": 4259662188, "table_properties": {"data_size": 34191264, "index_size": 1771264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26069988, "raw_average_key_size": 28, "raw_value_size": 108147818, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931071, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732366859, "oldest_key_time": 0, "file_creation_time": 1732366859, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280508, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:01:02.526151 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280509.sst) /dev/shm/ToplingTemp/Topling-188197-dRHgnK 2024/11/23-21:01:03.743665 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280509.sst): first pass time = 1.22's, 110.243'MB/sec 2024/11/23-21:01:05.223693 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280509.sst): old prealloc_size = 147639500, real_size = 36639312 2024/11/23-21:01:05.224410 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280509.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932992 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0346 GB avg-val = 115.86 avg-zval = 37.09 2024/11/23-21:01:05.225114 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280509.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:01:06.187767 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280509.sst): second pass time = 1.48's, 73.078'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 41.479'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2591.736'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.289'MB/sec write SST data time = 0.16's, 256.356'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36172.171'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.078'MB/sec zip pipeline throughput = 73.078'MB/sec entries = 932992 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.86 avg-zval = 37.09 usrkeys = 932992 avg-key = 20.00 avg-zkey = 1.93 avg-val = 120.56 avg-zval = 37.09 TagRS{ kinds = RS_Key0_TagN bytes = 145880 } TagArray{ size = 626200 bytes = 2348272 } seq expand size = 4383400 multi value expand size = 0 cnt WriteAppend = 21 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0346 GB dict = 1.08 MB all = 0.0400 GB } UnZip/Zip{ index = 10.3747 value = 3.1234 dict = 1.00 all = 3.3565 } Zip/UnZip{ index = 0.0964 value = 0.3202 dict = 1.00 all = 0.2979 } ---------------------------- total value len = 9.075763 GB avg = 0.167 KB (by entry num) total key len = 1.493015 GB avg = 0.028 KB total ukey len = 1.035159 GB avg = 0.015 KB total ukey num = 0.067414498 Billion total entry num = 0.054222227 Billion write speed all = 0.242659319 MB/sec (with seq num) write speed all = 0.232699772 MB/sec (without seq num) 2024/11/23-21:01:06.238700 139957668369984 [compaction_job.cc:1948] [default] [JOB 35738] Generated table #280509: 932992 keys, 39987690 bytes, temperature: kWarm 2024/11/23-21:01:06.238767 139957668369984 EVENT_LOG_v1 {"time_micros": 1732366866238732, "cf_name": "default", "job": 35738, "event": "table_file_creation", "file_number": 280509, "file_size": 39987690, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012775947, "table_properties": {"data_size": 34607296, "index_size": 1798592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26123776, "raw_average_key_size": 28, "raw_value_size": 108094084, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932992, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732366862, "oldest_key_time": 0, "file_creation_time": 1732366862, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280509, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:01:06.288652 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280510.sst) /dev/shm/ToplingTemp/Topling-188197-lUxhoL 2024/11/23-21:01:07.543325 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280510.sst): first pass time = 1.25's, 106.978'MB/sec 2024/11/23-21:01:09.151643 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280510.sst): old prealloc_size = 147639500, real_size = 36596784 2024/11/23-21:01:09.152344 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280510.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927472 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0346 GB avg-val = 116.71 avg-zval = 37.28 2024/11/23-21:01:09.153059 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280510.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:01:09.970403 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280510.sst): second pass time = 1.61's, 67.340'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 49.334'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2639.432'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.337'MB/sec write SST data time = 0.13's, 313.014'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35642.812'MB/sec zip my value time = 1.61's, unzip length = 0.108'GB zip my value throughput = 67.340'MB/sec zip pipeline throughput = 67.340'MB/sec entries = 927472 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.71 avg-zval = 37.28 usrkeys = 927472 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.33 avg-zval = 37.28 TagRS{ kinds = RS_Key0_TagN bytes = 145000 } TagArray{ size = 744475 bytes = 2791792 } seq expand size = 5211325 multi value expand size = 0 cnt WriteAppend = 21 UnZipSize{ index = 0.0185 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0346 GB dict = 1.08 MB all = 0.0404 GB } UnZip/Zip{ index = 10.3779 value = 3.1306 dict = 1.00 all = 3.3229 } Zip/UnZip{ index = 0.0964 value = 0.3194 dict = 1.00 all = 0.3009 } ---------------------------- total value len = 9.189223 GB avg = 0.167 KB (by entry num) total key len = 1.518984 GB avg = 0.028 KB total ukey len = 1.053709 GB avg = 0.015 KB total ukey num = 0.068341970 Billion total entry num = 0.055149699 Billion write speed all = 0.245839264 MB/sec (with seq num) write speed all = 0.235710239 MB/sec (without seq num) 2024/11/23-21:01:10.021647 139957668369984 [compaction_job.cc:1948] [default] [JOB 35738] Generated table #280510: 927472 keys, 40391610 bytes, temperature: kWarm 2024/11/23-21:01:10.021719 139957668369984 EVENT_LOG_v1 {"time_micros": 1732366870021684, "cf_name": "default", "job": 35738, "event": "table_file_creation", "file_number": 280510, "file_size": 40391610, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012775967, "table_properties": {"data_size": 34577312, "index_size": 1787392, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25969216, "raw_average_key_size": 28, "raw_value_size": 108248632, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927472, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732366866, "oldest_key_time": 0, "file_creation_time": 1732366866, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280510, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:01:10.058929 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280511.sst) /dev/shm/ToplingTemp/Topling-188197-l06wvL 2024/11/23-21:01:11.201306 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280511.sst): first pass time = 1.14's, 117.494'MB/sec 2024/11/23-21:01:12.661121 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280511.sst): old prealloc_size = 147639500, real_size = 32720544 2024/11/23-21:01:12.661754 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280511.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928905 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0307 GB avg-val = 116.49 avg-zval = 33.07 2024/11/23-21:01:12.662605 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280511.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:01:13.513904 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280511.sst): second pass time = 1.46's, 74.170'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.319'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2905.800'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 275.192'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 43652.030'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 74.170'MB/sec zip pipeline throughput = 74.170'MB/sec entries = 928905 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.49 avg-zval = 33.07 usrkeys = 928905 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.49 avg-zval = 33.07 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928905 bytes = 3367296 } seq expand size = 6502335 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5133 value = 3.5223 dict = 1.00 all = 3.6333 } Zip/UnZip{ index = 0.0951 value = 0.2839 dict = 1.00 all = 0.2752 } ---------------------------- total value len = 9.303933 GB avg = 0.166 KB (by entry num) total key len = 1.544993 GB avg = 0.028 KB total ukey len = 1.072287 GB avg = 0.015 KB total ukey num = 0.069270875 Billion total entry num = 0.056078604 Billion write speed all = 0.249049660 MB/sec (with seq num) write speed all = 0.238750866 MB/sec (without seq num) 2024/11/23-21:01:13.560746 139957668369984 [compaction_job.cc:1948] [default] [JOB 35738] Generated table #280511: 928905 keys, 36941058 bytes, temperature: kWarm 2024/11/23-21:01:13.560815 139957668369984 EVENT_LOG_v1 {"time_micros": 1732366873560778, "cf_name": "default", "job": 35738, "event": "table_file_creation", "file_number": 280511, "file_size": 36941058, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776210, "table_properties": {"data_size": 30721040, "index_size": 1767104, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26009340, "raw_average_key_size": 28, "raw_value_size": 108208490, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928905, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732366870, "oldest_key_time": 0, "file_creation_time": 1732366870, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280511, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:01:13.597519 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280512.sst) /dev/shm/ToplingTemp/Topling-188197-2x10fN 2024/11/23-21:01:14.857483 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280512.sst): first pass time = 1.26's, 106.529'MB/sec 2024/11/23-21:01:16.363989 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280512.sst): old prealloc_size = 147639500, real_size = 32516576 2024/11/23-21:01:16.364667 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280512.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932232 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0305 GB avg-val = 115.97 avg-zval = 32.72 2024/11/23-21:01:16.365506 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280512.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:01:17.224246 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280512.sst): second pass time = 1.50's, 71.839'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.716'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2763.249'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.13's, 278.365'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35248.889'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 71.839'MB/sec zip pipeline throughput = 71.839'MB/sec entries = 932232 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.97 avg-zval = 32.72 usrkeys = 932232 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.97 avg-zval = 32.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932232 bytes = 3379360 } seq expand size = 6525624 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.4458 value = 3.5449 dict = 1.00 all = 3.6525 } Zip/UnZip{ index = 0.0957 value = 0.2821 dict = 1.00 all = 0.2738 } ---------------------------- total value len = 9.418574 GB avg = 0.165 KB (by entry num) total key len = 1.571096 GB avg = 0.028 KB total ukey len = 1.090931 GB avg = 0.016 KB total ukey num = 0.070203107 Billion total entry num = 0.057010836 Billion write speed all = 0.252259103 MB/sec (with seq num) write speed all = 0.241789997 MB/sec (without seq num) 2024/11/23-21:01:17.272330 139957668369984 [compaction_job.cc:1948] [default] [JOB 35738] Generated table #280512: 932232 keys, 36746958 bytes, temperature: kWarm 2024/11/23-21:01:17.272395 139957668369984 EVENT_LOG_v1 {"time_micros": 1732366877272362, "cf_name": "default", "job": 35738, "event": "table_file_creation", "file_number": 280512, "file_size": 36746958, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776248, "table_properties": {"data_size": 30498448, "index_size": 1784896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26102496, "raw_average_key_size": 28, "raw_value_size": 108115352, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932232, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732366873, "oldest_key_time": 0, "file_creation_time": 1732366873, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280512, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:01:17.311868 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280513.sst) /dev/shm/ToplingTemp/Topling-188197-qDrfgO 2024/11/23-21:01:18.494152 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280513.sst): first pass time = 1.18's, 113.528'MB/sec 2024/11/23-21:01:19.998955 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280513.sst): old prealloc_size = 147639500, real_size = 34099552 2024/11/23-21:01:19.999698 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280513.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933600 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0321 GB avg-val = 115.76 avg-zval = 34.36 2024/11/23-21:01:20.000598 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280513.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:01:20.859796 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280513.sst): second pass time = 1.50's, 71.917'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 44.668'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2579.557'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.325'MB/sec write SST data time = 0.14's, 271.548'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36350.568'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 71.917'MB/sec zip pipeline throughput = 71.917'MB/sec entries = 933600 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.76 avg-zval = 34.36 usrkeys = 933600 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.76 avg-zval = 34.36 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933600 bytes = 3501008 } seq expand size = 6535200 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0321 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.4435 value = 3.3692 dict = 1.00 all = 3.4906 } Zip/UnZip{ index = 0.0958 value = 0.2968 dict = 1.00 all = 0.2865 } ---------------------------- total value len = 9.533186 GB avg = 0.165 KB (by entry num) total key len = 1.597237 GB avg = 0.028 KB total ukey len = 1.109603 GB avg = 0.016 KB total ukey num = 0.071136707 Billion total entry num = 0.057944436 Billion write speed all = 0.255468655 MB/sec (with seq num) write speed all = 0.244828997 MB/sec (without seq num) 2024/11/23-21:01:20.910260 139957668369984 [compaction_job.cc:1948] [default] [JOB 35738] Generated table #280513: 933600 keys, 38450894 bytes, temperature: kWarm 2024/11/23-21:01:20.910320 139957668369984 EVENT_LOG_v1 {"time_micros": 1732366880910285, "cf_name": "default", "job": 35738, "event": "table_file_creation", "file_number": 280513, "file_size": 38450894, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776268, "table_properties": {"data_size": 32078096, "index_size": 1787904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26140800, "raw_average_key_size": 28, "raw_value_size": 108076936, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933600, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732366877, "oldest_key_time": 0, "file_creation_time": 1732366877, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280513, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:01:20.953734 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280514.sst) /dev/shm/ToplingTemp/Topling-188197-GRup7J 2024/11/23-21:01:22.128312 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280514.sst): first pass time = 1.17's, 114.273'MB/sec 2024/11/23-21:01:23.654954 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280514.sst): old prealloc_size = 147639500, real_size = 30529184 2024/11/23-21:01:23.655682 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280514.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928557 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0285 GB avg-val = 116.54 avg-zval = 30.70 2024/11/23-21:01:23.657510 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280514.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:01:24.506712 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280514.sst): second pass time = 1.52's, 71.227'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 40.932'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2585.132'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.333'MB/sec write SST data time = 0.15's, 227.148'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32983.689'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 71.227'MB/sec zip pipeline throughput = 71.227'MB/sec entries = 928557 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.54 avg-zval = 30.70 usrkeys = 928557 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.54 avg-zval = 30.70 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928557 bytes = 3482096 } seq expand size = 6499899 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0285 GB dict = 1.08 MB all = 0.0349 GB } UnZip/Zip{ index = 10.3760 value = 3.7962 dict = 1.00 all = 3.8499 } Zip/UnZip{ index = 0.0964 value = 0.2634 dict = 1.00 all = 0.2597 } ---------------------------- total value len = 9.647905 GB avg = 0.164 KB (by entry num) total key len = 1.623236 GB avg = 0.028 KB total ukey len = 1.128175 GB avg = 0.016 KB total ukey num = 0.072065264 Billion total entry num = 0.058872993 Billion write speed all = 0.258676795 MB/sec (with seq num) write speed all = 0.247867541 MB/sec (without seq num) 2024/11/23-21:01:24.551686 139957668369984 [compaction_job.cc:1948] [default] [JOB 35738] Generated table #280514: 928557 keys, 34862876 bytes, temperature: kWarm 2024/11/23-21:01:24.551753 139957668369984 EVENT_LOG_v1 {"time_micros": 1732366884551719, "cf_name": "default", "job": 35738, "event": "table_file_creation", "file_number": 280514, "file_size": 34862876, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3414597306, "largest_seqno": 4012776300, "table_properties": {"data_size": 28507024, "index_size": 1789824, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25999596, "raw_average_key_size": 28, "raw_value_size": 108218178, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928557, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732366880, "oldest_key_time": 0, "file_creation_time": 1732366880, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280514, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:01:24.590182 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280515.sst) /dev/shm/ToplingTemp/Topling-188197-sYHyZM 2024/11/23-21:01:25.780757 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280515.sst): first pass time = 1.19's, 112.737'MB/sec 2024/11/23-21:01:27.344884 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280515.sst): old prealloc_size = 147639500, real_size = 35292640 2024/11/23-21:01:27.345462 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280515.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925382 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0333 GB avg-val = 117.04 avg-zval = 35.97 2024/11/23-21:01:27.346281 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280515.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:01:28.198262 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280515.sst): second pass time = 1.56's, 69.636'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 46.426'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3152.635'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 310.996'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 27516.841'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.636'MB/sec zip pipeline throughput = 69.636'MB/sec entries = 925382 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.04 avg-zval = 35.97 usrkeys = 925382 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.04 avg-zval = 35.97 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925382 bytes = 3470192 } seq expand size = 6477674 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0333 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.4451 value = 3.2535 dict = 1.00 all = 3.3878 } Zip/UnZip{ index = 0.0957 value = 0.3074 dict = 1.00 all = 0.2952 } ---------------------------- total value len = 9.762689 GB avg = 0.163 KB (by entry num) total key len = 1.649147 GB avg = 0.028 KB total ukey len = 1.146682 GB avg = 0.016 KB total ukey num = 0.072990646 Billion total entry num = 0.059798375 Billion write speed all = 0.261883619 MB/sec (with seq num) write speed all = 0.250905393 MB/sec (without seq num) 2024/11/23-21:01:28.246647 139957668369984 [compaction_job.cc:1948] [default] [JOB 35738] Generated table #280515: 925382 keys, 39618130 bytes, temperature: kWarm 2024/11/23-21:01:28.246714 139957668369984 EVENT_LOG_v1 {"time_micros": 1732366888246681, "cf_name": "default", "job": 35738, "event": "table_file_creation", "file_number": 280515, "file_size": 39618130, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776316, "table_properties": {"data_size": 33289232, "index_size": 1771904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25910696, "raw_average_key_size": 28, "raw_value_size": 108307076, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925382, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732366884, "oldest_key_time": 0, "file_creation_time": 1732366884, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280515, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:01:28.290385 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280516.sst) /dev/shm/ToplingTemp/Topling-188197-N63atL 2024/11/23-21:01:29.434920 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280516.sst): first pass time = 1.14's, 117.273'MB/sec 2024/11/23-21:01:30.904607 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280516.sst): old prealloc_size = 147639500, real_size = 31305856 2024/11/23-21:01:30.905361 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280516.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931083 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0293 GB avg-val = 116.15 avg-zval = 31.47 2024/11/23-21:01:30.907209 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280516.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:01:31.772660 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280516.sst): second pass time = 1.47's, 73.626'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 41.068'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2429.994'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.325'MB/sec write SST data time = 0.15's, 236.002'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 27860.615'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.626'MB/sec zip pipeline throughput = 73.626'MB/sec entries = 931083 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.15 avg-zval = 31.47 usrkeys = 931083 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.15 avg-zval = 31.47 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931083 bytes = 3491584 } seq expand size = 6517581 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.08 MB all = 0.0356 GB } UnZip/Zip{ index = 10.5132 value = 3.6908 dict = 1.00 all = 3.7650 } Zip/UnZip{ index = 0.0951 value = 0.2709 dict = 1.00 all = 0.2656 } ---------------------------- total value len = 9.877354 GB avg = 0.163 KB (by entry num) total key len = 1.675217 GB avg = 0.028 KB total ukey len = 1.165304 GB avg = 0.016 KB total ukey num = 0.073921729 Billion total entry num = 0.060729458 Billion write speed all = 0.265091530 MB/sec (with seq num) write speed all = 0.253943284 MB/sec (without seq num) 2024/11/23-21:01:31.818415 139957668369984 [compaction_job.cc:1948] [default] [JOB 35738] Generated table #280516: 931083 keys, 35649080 bytes, temperature: kWarm 2024/11/23-21:01:31.818478 139957668369984 EVENT_LOG_v1 {"time_micros": 1732366891818441, "cf_name": "default", "job": 35738, "event": "table_file_creation", "file_number": 280516, "file_size": 35649080, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012776344, "table_properties": {"data_size": 29301616, "index_size": 1771264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26070324, "raw_average_key_size": 28, "raw_value_size": 108147522, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931083, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732366888, "oldest_key_time": 0, "file_creation_time": 1732366888, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280516, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:01:31.855608 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280517.sst) /dev/shm/ToplingTemp/Topling-188197-BPrupL 2024/11/23-21:01:32.859164 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280517.sst): first pass time = 1.00's, 133.747'MB/sec 2024/11/23-21:01:34.345242 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280517.sst): old prealloc_size = 147639500, real_size = 36809984 2024/11/23-21:01:34.345781 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280517.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924764 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0348 GB avg-val = 117.14 avg-zval = 37.63 2024/11/23-21:01:34.346143 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280517.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:01:35.070400 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280517.sst): second pass time = 1.49's, 72.934'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.73's, 53.518'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3468.456'MB/sec (index lex order gen) rebuild zvType time = 0.60's, 0.385'MB/sec write SST data time = 0.12's, 313.658'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33666.054'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.934'MB/sec zip pipeline throughput = 72.934'MB/sec entries = 924764 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.14 avg-zval = 37.63 usrkeys = 924764 avg-key = 20.00 avg-zkey = 1.93 avg-val = 119.21 avg-zval = 37.63 TagRS{ kinds = RS_Key0_TagN bytes = 144600 } TagArray{ size = 273800 bytes = 992544 } seq expand size = 1916600 multi value expand size = 0 cnt WriteAppend = 20 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0348 GB dict = 1.08 MB all = 0.0388 GB } UnZip/Zip{ index = 10.3762 value = 3.1131 dict = 1.00 all = 3.4585 } Zip/UnZip{ index = 0.0964 value = 0.3212 dict = 1.00 all = 0.2891 } ---------------------------- total value len = 9.987595 GB avg = 0.162 KB (by entry num) total key len = 1.701111 GB avg = 0.028 KB total ukey len = 1.183799 GB avg = 0.016 KB total ukey num = 0.074846493 Billion total entry num = 0.061654222 Billion write speed all = 0.268195049 MB/sec (with seq num) write speed all = 0.256877898 MB/sec (without seq num) 2024/11/23-21:01:35.119958 139957668369984 [compaction_job.cc:1948] [default] [JOB 35738] Generated table #280517: 924764 keys, 38807735 bytes, temperature: kWarm 2024/11/23-21:01:35.120050 139957668369984 EVENT_LOG_v1 {"time_micros": 1732366895120007, "cf_name": "default", "job": 35738, "event": "table_file_creation", "file_number": 280517, "file_size": 38807735, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3680844404, "table_properties": {"data_size": 34796144, "index_size": 1782464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25893392, "raw_average_key_size": 28, "raw_value_size": 108324428, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924764, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732366891, "oldest_key_time": 0, "file_creation_time": 1732366891, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280517, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:01:35.155281 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280518.sst) /dev/shm/ToplingTemp/Topling-188197-WKeUcM 2024/11/23-21:01:36.192005 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280518.sst): first pass time = 1.04's, 129.468'MB/sec 2024/11/23-21:01:37.633872 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280518.sst): old prealloc_size = 147639500, real_size = 36062832 2024/11/23-21:01:37.634435 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280518.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927324 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0341 GB avg-val = 116.74 avg-zval = 36.74 2024/11/23-21:01:37.634992 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280518.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:01:38.411339 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280518.sst): second pass time = 1.44's, 75.119'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.78's, 50.343'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3278.989'MB/sec (index lex order gen) rebuild zvType time = 0.65's, 0.356'MB/sec write SST data time = 0.13's, 309.506'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31149.818'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.119'MB/sec zip pipeline throughput = 75.119'MB/sec entries = 927324 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.74 avg-zval = 36.74 usrkeys = 927324 avg-key = 20.00 avg-zkey = 1.90 avg-val = 120.91 avg-zval = 36.74 TagRS{ kinds = RS_Key0_TagN bytes = 145000 } TagArray{ size = 552719 bytes = 2072704 } seq expand size = 3869033 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0341 GB dict = 1.08 MB all = 0.0391 GB } UnZip/Zip{ index = 10.5133 value = 3.1777 dict = 1.00 all = 3.4293 } Zip/UnZip{ index = 0.0951 value = 0.3147 dict = 1.00 all = 0.2916 } ---------------------------- total value len = 10.099717 GB avg = 0.161 KB (by entry num) total key len = 1.727076 GB avg = 0.028 KB total ukey len = 1.202346 GB avg = 0.016 KB total ukey num = 0.075773817 Billion total entry num = 0.062581546 Billion write speed all = 0.271342625 MB/sec (with seq num) write speed all = 0.259856136 MB/sec (without seq num) 2024/11/23-21:01:38.458623 139957668369984 [compaction_job.cc:1948] [default] [JOB 35738] Generated table #280518: 927324 keys, 39138594 bytes, temperature: kWarm 2024/11/23-21:01:38.458693 139957668369984 EVENT_LOG_v1 {"time_micros": 1732366898458656, "cf_name": "default", "job": 35738, "event": "table_file_creation", "file_number": 280518, "file_size": 39138594, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4259661717, "table_properties": {"data_size": 34066720, "index_size": 1764096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25965072, "raw_average_key_size": 28, "raw_value_size": 108252788, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927324, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732366895, "oldest_key_time": 0, "file_creation_time": 1732366895, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280518, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:01:38.500049 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280519.sst) /dev/shm/ToplingTemp/Topling-188197-SvQ0WM 2024/11/23-21:01:39.540652 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280519.sst): first pass time = 1.04's, 128.985'MB/sec 2024/11/23-21:01:40.867877 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280519.sst): old prealloc_size = 147639500, real_size = 33797360 2024/11/23-21:01:40.868437 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280519.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927832 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0318 GB avg-val = 116.66 avg-zval = 34.27 2024/11/23-21:01:40.869235 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280519.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:01:41.710754 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280519.sst): second pass time = 1.33's, 81.602'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.965'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3294.465'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.12's, 304.384'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33984.710'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.602'MB/sec zip pipeline throughput = 81.602'MB/sec entries = 927832 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.66 avg-zval = 34.27 usrkeys = 927832 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.66 avg-zval = 34.27 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927832 bytes = 3247424 } seq expand size = 6494824 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.5133 value = 3.4037 dict = 1.00 all = 3.5415 } Zip/UnZip{ index = 0.0951 value = 0.2938 dict = 1.00 all = 0.2824 } ---------------------------- total value len = 10.214451 GB avg = 0.161 KB (by entry num) total key len = 1.753055 GB avg = 0.028 KB total ukey len = 1.220902 GB avg = 0.016 KB total ukey num = 0.076701649 Billion total entry num = 0.063509378 Billion write speed all = 0.274550218 MB/sec (with seq num) write speed all = 0.262894313 MB/sec (without seq num) 2024/11/23-21:01:41.758374 139957668369984 [compaction_job.cc:1948] [default] [JOB 35738] Generated table #280519: 927832 keys, 37898908 bytes, temperature: kWarm 2024/11/23-21:01:41.758446 139957668369984 EVENT_LOG_v1 {"time_micros": 1732366901758407, "cf_name": "default", "job": 35738, "event": "table_file_creation", "file_number": 280519, "file_size": 37898908, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4100102034, "largest_seqno": 4259661598, "table_properties": {"data_size": 31800160, "index_size": 1765056, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25979296, "raw_average_key_size": 28, "raw_value_size": 108238488, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927832, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732366898, "oldest_key_time": 0, "file_creation_time": 1732366898, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280519, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:01:41.794565 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280520.sst) /dev/shm/ToplingTemp/Topling-188197-BVov4K 2024/11/23-21:01:42.837436 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280520.sst): first pass time = 1.04's, 128.706'MB/sec 2024/11/23-21:01:44.162614 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280520.sst): old prealloc_size = 147639500, real_size = 34097424 2024/11/23-21:01:44.163178 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280520.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928254 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0321 GB avg-val = 116.59 avg-zval = 34.57 2024/11/23-21:01:44.163898 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280520.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:01:45.007607 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280520.sst): second pass time = 1.32's, 81.717'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.934'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3292.498'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.12's, 304.872'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 28113.289'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.717'MB/sec zip pipeline throughput = 81.717'MB/sec entries = 928254 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.59 avg-zval = 34.57 usrkeys = 928254 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.59 avg-zval = 34.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928254 bytes = 3016848 } seq expand size = 6497778 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0321 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.4439 value = 3.3729 dict = 1.00 all = 3.5350 } Zip/UnZip{ index = 0.0957 value = 0.2965 dict = 1.00 all = 0.2829 } ---------------------------- total value len = 10.329175 GB avg = 0.160 KB (by entry num) total key len = 1.779046 GB avg = 0.028 KB total ukey len = 1.239467 GB avg = 0.016 KB total ukey num = 0.077629903 Billion total entry num = 0.064437632 Billion write speed all = 0.277757410 MB/sec (with seq num) write speed all = 0.265932037 MB/sec (without seq num) 2024/11/23-21:01:45.054858 139957668369984 [compaction_job.cc:1948] [default] [JOB 35738] Generated table #280520: 928254 keys, 37967988 bytes, temperature: kWarm 2024/11/23-21:01:45.054930 139957668369984 EVENT_LOG_v1 {"time_micros": 1732366905054894, "cf_name": "default", "job": 35738, "event": "table_file_creation", "file_number": 280520, "file_size": 37967988, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4199074359, "largest_seqno": 4259661895, "table_properties": {"data_size": 32087552, "index_size": 1777600, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25991112, "raw_average_key_size": 28, "raw_value_size": 108226732, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928254, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732366901, "oldest_key_time": 0, "file_creation_time": 1732366901, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280520, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:01:45.088682 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280521.sst) /dev/shm/ToplingTemp/Topling-188197-hS8X9L 2024/11/23-21:01:45.949134 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280521.sst): first pass time = 0.86's, 155.991'MB/sec 2024/11/23-21:01:47.275101 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280521.sst): old prealloc_size = 147639500, real_size = 34340000 2024/11/23-21:01:47.275615 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280521.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928446 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0323 GB avg-val = 116.56 avg-zval = 34.83 2024/11/23-21:01:47.276431 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280521.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:01:48.114701 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280521.sst): second pass time = 1.33's, 81.666'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 45.788'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3599.797'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.13's, 306.644'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 31411.902'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.666'MB/sec zip pipeline throughput = 81.666'MB/sec entries = 928446 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.56 avg-zval = 34.83 usrkeys = 928446 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.56 avg-zval = 34.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928446 bytes = 3249568 } seq expand size = 6499122 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0323 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.5134 value = 3.3462 dict = 1.00 all = 3.4914 } Zip/UnZip{ index = 0.0951 value = 0.2988 dict = 1.00 all = 0.2864 } ---------------------------- total value len = 10.443896 GB avg = 0.160 KB (by entry num) total key len = 1.805043 GB avg = 0.028 KB total ukey len = 1.258036 GB avg = 0.016 KB total ukey num = 0.078558349 Billion total entry num = 0.065366078 Billion write speed all = 0.280965370 MB/sec (with seq num) write speed all = 0.268970466 MB/sec (without seq num) 2024/11/23-21:01:48.161079 139957668369984 [compaction_job.cc:1948] [default] [JOB 35738] Generated table #280521: 928446 keys, 38442750 bytes, temperature: kWarm 2024/11/23-21:01:48.161152 139957668369984 EVENT_LOG_v1 {"time_micros": 1732366908161115, "cf_name": "default", "job": 35738, "event": "table_file_creation", "file_number": 280521, "file_size": 38442750, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4116739076, "largest_seqno": 4259661790, "table_properties": {"data_size": 32341520, "index_size": 1766208, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25996488, "raw_average_key_size": 28, "raw_value_size": 108221304, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928446, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732366905, "oldest_key_time": 0, "file_creation_time": 1732366905, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280521, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:01:48.213553 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280522.sst) /dev/shm/ToplingTemp/Topling-188197-UFtUhN 2024/11/23-21:01:49.186499 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280522.sst): first pass time = 0.97's, 137.957'MB/sec 2024/11/23-21:01:50.469666 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280522.sst): old prealloc_size = 147639500, real_size = 34350448 2024/11/23-21:01:50.470240 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280522.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932799 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0323 GB avg-val = 115.89 avg-zval = 34.67 2024/11/23-21:01:50.470983 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280522.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:01:51.308718 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280522.sst): second pass time = 1.28's, 84.318'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 45.711'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3210.228'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.328'MB/sec write SST data time = 0.13's, 298.733'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34204.448'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.318'MB/sec zip pipeline throughput = 84.318'MB/sec entries = 932799 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.89 avg-zval = 34.67 usrkeys = 932799 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.89 avg-zval = 34.67 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932799 bytes = 3148208 } seq expand size = 6529593 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0323 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.5124 value = 3.3423 dict = 1.00 all = 3.4995 } Zip/UnZip{ index = 0.0951 value = 0.2992 dict = 1.00 all = 0.2858 } ---------------------------- total value len = 10.558524 GB avg = 0.159 KB (by entry num) total key len = 1.831161 GB avg = 0.028 KB total ukey len = 1.276692 GB avg = 0.016 KB total ukey num = 0.079491148 Billion total entry num = 0.066298877 Billion write speed all = 0.284173004 MB/sec (with seq num) write speed all = 0.272007820 MB/sec (without seq num) 2024/11/23-21:01:51.355380 139957668369984 [compaction_job.cc:1948] [default] [JOB 35738] Generated table #280522: 932799 keys, 38353178 bytes, temperature: kWarm 2024/11/23-21:01:51.355452 139957668369984 EVENT_LOG_v1 {"time_micros": 1732366911355415, "cf_name": "default", "job": 35738, "event": "table_file_creation", "file_number": 280522, "file_size": 38353178, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4159804009, "largest_seqno": 4259661869, "table_properties": {"data_size": 32342432, "index_size": 1774656, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26118372, "raw_average_key_size": 28, "raw_value_size": 108099378, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932799, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732366908, "oldest_key_time": 0, "file_creation_time": 1732366908, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280522, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:01:51.391194 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280523.sst) /dev/shm/ToplingTemp/Topling-188197-Clir6J 2024/11/23-21:01:52.422300 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280523.sst): first pass time = 1.03's, 130.174'MB/sec 2024/11/23-21:01:53.780474 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280523.sst): old prealloc_size = 147639500, real_size = 33947824 2024/11/23-21:01:53.781069 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280523.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929587 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0319 GB avg-val = 116.38 avg-zval = 34.35 2024/11/23-21:01:53.781810 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280523.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:01:54.627460 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280523.sst): second pass time = 1.36's, 79.703'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.793'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3118.295'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 300.471'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33562.729'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.703'MB/sec zip pipeline throughput = 79.703'MB/sec entries = 929587 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.38 avg-zval = 34.35 usrkeys = 929587 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.38 avg-zval = 34.35 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929587 bytes = 3137376 } seq expand size = 6507109 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.4446 value = 3.3878 dict = 1.00 all = 3.5378 } Zip/UnZip{ index = 0.0957 value = 0.2952 dict = 1.00 all = 0.2827 } ---------------------------- total value len = 10.673221 GB avg = 0.159 KB (by entry num) total key len = 1.857189 GB avg = 0.028 KB total ukey len = 1.295284 GB avg = 0.016 KB total ukey num = 0.080420735 Billion total entry num = 0.067228464 Billion write speed all = 0.287378831 MB/sec (with seq num) write speed all = 0.275044016 MB/sec (without seq num) 2024/11/23-21:01:54.672862 139957668369984 [compaction_job.cc:1948] [default] [JOB 35738] Generated table #280523: 929587 keys, 37937816 bytes, temperature: kWarm 2024/11/23-21:01:54.672932 139957668369984 EVENT_LOG_v1 {"time_micros": 1732366914672896, "cf_name": "default", "job": 35738, "event": "table_file_creation", "file_number": 280523, "file_size": 37937816, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4142233521, "largest_seqno": 4259661907, "table_properties": {"data_size": 31935200, "index_size": 1780032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26028436, "raw_average_key_size": 28, "raw_value_size": 108189354, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929587, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732366911, "oldest_key_time": 0, "file_creation_time": 1732366911, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280523, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:01:54.708995 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280524.sst) /dev/shm/ToplingTemp/Topling-188197-XN33gL 2024/11/23-21:01:55.738040 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280524.sst): first pass time = 1.03's, 130.435'MB/sec 2024/11/23-21:01:57.039999 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280524.sst): old prealloc_size = 147639500, real_size = 32822432 2024/11/23-21:01:57.040557 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280524.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928309 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0308 GB avg-val = 116.58 avg-zval = 33.20 2024/11/23-21:01:57.041323 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280524.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:01:57.884859 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280524.sst): second pass time = 1.30's, 83.176'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.568'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3311.293'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 291.920'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33160.636'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.176'MB/sec zip pipeline throughput = 83.176'MB/sec entries = 928309 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.58 avg-zval = 33.20 usrkeys = 928309 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.58 avg-zval = 33.20 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928309 bytes = 3133056 } seq expand size = 6498163 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5146 value = 3.5110 dict = 1.00 all = 3.6463 } Zip/UnZip{ index = 0.0951 value = 0.2848 dict = 1.00 all = 0.2742 } ---------------------------- total value len = 10.787944 GB avg = 0.158 KB (by entry num) total key len = 1.883182 GB avg = 0.028 KB total ukey len = 1.313850 GB avg = 0.016 KB total ukey num = 0.081349044 Billion total entry num = 0.068156773 Billion write speed all = 0.290584374 MB/sec (with seq num) write speed all = 0.278080170 MB/sec (without seq num) 2024/11/23-21:01:57.930576 139957668369984 [compaction_job.cc:1948] [default] [JOB 35738] Generated table #280524: 928309 keys, 36808876 bytes, temperature: kWarm 2024/11/23-21:01:57.930650 139957668369984 EVENT_LOG_v1 {"time_micros": 1732366917930612, "cf_name": "default", "job": 35738, "event": "table_file_creation", "file_number": 280524, "file_size": 36808876, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4179164037, "largest_seqno": 4259661922, "table_properties": {"data_size": 30824400, "index_size": 1765760, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25992652, "raw_average_key_size": 28, "raw_value_size": 108225134, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928309, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732366914, "oldest_key_time": 0, "file_creation_time": 1732366914, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280524, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:01:57.966274 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280525.sst) /dev/shm/ToplingTemp/Topling-188197-9nD2AM 2024/11/23-21:01:58.989057 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280525.sst): first pass time = 1.02's, 131.234'MB/sec 2024/11/23-21:02:00.301013 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280525.sst): old prealloc_size = 147639500, real_size = 32951056 2024/11/23-21:02:00.301552 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280525.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930246 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.28 avg-zval = 33.27 2024/11/23-21:02:00.302317 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280525.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:02:01.160526 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280525.sst): second pass time = 1.31's, 82.501'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.845'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3516.025'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 291.467'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33967.134'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.501'MB/sec zip pipeline throughput = 82.501'MB/sec entries = 930246 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.28 avg-zval = 33.27 usrkeys = 930246 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.28 avg-zval = 33.27 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930246 bytes = 3023312 } seq expand size = 6511722 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5129 value = 3.4952 dict = 1.00 all = 3.6446 } Zip/UnZip{ index = 0.0951 value = 0.2861 dict = 1.00 all = 0.2744 } ---------------------------- total value len = 10.902627 GB avg = 0.158 KB (by entry num) total key len = 1.909229 GB avg = 0.028 KB total ukey len = 1.332455 GB avg = 0.016 KB total ukey num = 0.082279290 Billion total entry num = 0.069087019 Billion write speed all = 0.293789626 MB/sec (with seq num) write speed all = 0.281115710 MB/sec (without seq num) 2024/11/23-21:02:01.206447 139957668369984 [compaction_job.cc:1948] [default] [JOB 35738] Generated table #280525: 930246 keys, 36826630 bytes, temperature: kWarm 2024/11/23-21:02:01.206516 139957668369984 EVENT_LOG_v1 {"time_micros": 1732366921206480, "cf_name": "default", "job": 35738, "event": "table_file_creation", "file_number": 280525, "file_size": 36826630, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4201818573, "largest_seqno": 4259661950, "table_properties": {"data_size": 30948608, "index_size": 1769728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26046888, "raw_average_key_size": 28, "raw_value_size": 108170928, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930246, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732366917, "oldest_key_time": 0, "file_creation_time": 1732366917, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280525, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:02:01.252297 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280526.sst) /dev/shm/ToplingTemp/Topling-188197-JxbBAL 2024/11/23-21:02:02.078154 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280526.sst): first pass time = 0.83's, 162.529'MB/sec 2024/11/23-21:02:03.408610 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280526.sst): old prealloc_size = 147639500, real_size = 32868544 2024/11/23-21:02:03.409211 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280526.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929737 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.36 avg-zval = 33.20 2024/11/23-21:02:03.410615 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280526.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:02:04.247671 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280526.sst): second pass time = 1.33's, 81.411'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.069'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3059.929'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 289.533'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32930.298'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.411'MB/sec zip pipeline throughput = 81.411'MB/sec entries = 929737 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.36 avg-zval = 33.20 usrkeys = 929737 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.36 avg-zval = 33.20 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929737 bytes = 3254096 } seq expand size = 6508159 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5117 value = 3.5049 dict = 1.00 all = 3.6299 } Zip/UnZip{ index = 0.0951 value = 0.2853 dict = 1.00 all = 0.2755 } ---------------------------- total value len = 11.017320 GB avg = 0.157 KB (by entry num) total key len = 1.935262 GB avg = 0.028 KB total ukey len = 1.351050 GB avg = 0.016 KB total ukey num = 0.083209027 Billion total entry num = 0.070016756 Billion write speed all = 0.296995601 MB/sec (with seq num) write speed all = 0.284152034 MB/sec (without seq num) 2024/11/23-21:02:04.295949 139957668369984 [compaction_job.cc:1948] [default] [JOB 35738] Generated table #280526: 929737 keys, 36975140 bytes, temperature: kWarm 2024/11/23-21:02:04.296037 139957668369984 EVENT_LOG_v1 {"time_micros": 1732366924295994, "cf_name": "default", "job": 35738, "event": "table_file_creation", "file_number": 280526, "file_size": 36975140, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4098574454, "largest_seqno": 4259661962, "table_properties": {"data_size": 30866992, "index_size": 1768960, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26032636, "raw_average_key_size": 28, "raw_value_size": 108185234, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929737, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732366921, "oldest_key_time": 0, "file_creation_time": 1732366921, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280526, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:02:04.334052 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280527.sst) /dev/shm/ToplingTemp/Topling-188197-UrffdL 2024/11/23-21:02:05.299553 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280527.sst): first pass time = 0.97's, 139.019'MB/sec 2024/11/23-21:02:06.662388 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280527.sst): old prealloc_size = 147639500, real_size = 33885216 2024/11/23-21:02:06.662901 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280527.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931607 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0319 GB avg-val = 116.07 avg-zval = 34.20 2024/11/23-21:02:06.663708 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280527.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:02:07.510260 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280527.sst): second pass time = 1.36's, 79.396'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.816'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3664.311'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 301.760'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34815.454'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.396'MB/sec zip pipeline throughput = 79.396'MB/sec entries = 931607 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.07 avg-zval = 34.20 usrkeys = 931607 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.07 avg-zval = 34.20 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931607 bytes = 3260640 } seq expand size = 6521249 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.3763 value = 3.3944 dict = 1.00 all = 3.5324 } Zip/UnZip{ index = 0.0964 value = 0.2946 dict = 1.00 all = 0.2831 } ---------------------------- total value len = 11.131974 GB avg = 0.157 KB (by entry num) total key len = 1.961347 GB avg = 0.028 KB total ukey len = 1.369682 GB avg = 0.016 KB total ukey num = 0.084140634 Billion total entry num = 0.070948363 Billion write speed all = 0.300200213 MB/sec (with seq num) write speed all = 0.287186730 MB/sec (without seq num) 2024/11/23-21:02:07.557736 139957668369984 [compaction_job.cc:1948] [default] [JOB 35738] Generated table #280527: 931607 keys, 37996582 bytes, temperature: kWarm 2024/11/23-21:02:07.557804 139957668369984 EVENT_LOG_v1 {"time_micros": 1732366927557769, "cf_name": "default", "job": 35738, "event": "table_file_creation", "file_number": 280527, "file_size": 37996582, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4042106349, "largest_seqno": 4259662036, "table_properties": {"data_size": 31856464, "index_size": 1795648, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26084996, "raw_average_key_size": 28, "raw_value_size": 108132858, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931607, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732366924, "oldest_key_time": 0, "file_creation_time": 1732366924, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280527, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:02:07.597933 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280528.sst) /dev/shm/ToplingTemp/Topling-188197-OMh1PN 2024/11/23-21:02:08.423406 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280528.sst): first pass time = 0.83's, 132.675'MB/sec 2024/11/23-21:02:09.507756 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280528.sst): old prealloc_size = 147639500, real_size = 27564208 2024/11/23-21:02:09.508246 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280528.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 759632 prefix = 4 raw-key = 0.0122 GB zip-key = 0.0014 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0882 GB zip-val = 0.0259 GB avg-val = 116.17 avg-zval = 34.13 2024/11/23-21:02:09.508824 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280528.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0019 GB, waited 0.000 sec, Key+Value = 0.110 GB 2024/11/23-21:02:10.203809 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00/280528.sst): second pass time = 1.08's, 81.465'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.70's, 44.016'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3110.130'MB/sec (index lex order gen) rebuild zvType time = 0.59's, 0.321'MB/sec write SST data time = 0.10's, 295.204'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.883'MB, throughput = 29257.096'MB/sec zip my value time = 1.08's, unzip length = 0.088'GB zip my value throughput = 81.465'MB/sec zip pipeline throughput = 81.465'MB/sec entries = 759632 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.17 avg-zval = 34.13 usrkeys = 759632 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.17 avg-zval = 34.13 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 759632 bytes = 2373872 } seq expand size = 5317424 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0152 GB value = 0.0882 GB dict = 0.88 MB all = 0.1095 GB } __ZipSize{ index = 0.0014 GB value = 0.0259 GB dict = 0.88 MB all = 0.0306 GB } UnZip/Zip{ index = 10.5126 value = 3.4033 dict = 1.00 all = 3.5746 } Zip/UnZip{ index = 0.0951 value = 0.2938 dict = 1.00 all = 0.2797 } ---------------------------- total value len = 11.225537 GB avg = 0.157 KB (by entry num) total key len = 1.982616 GB avg = 0.028 KB total ukey len = 1.384875 GB avg = 0.016 KB total ukey num = 0.084900266 Billion total entry num = 0.071707995 Billion write speed all = 0.302814353 MB/sec (with seq num) write speed all = 0.289662349 MB/sec (without seq num) 2024/11/23-21:02:10.242834 139957668369984 [compaction_job.cc:1948] [default] [JOB 35738] Generated table #280528: 759632 keys, 30636586 bytes, temperature: kWarm 2024/11/23-21:02:10.242905 139957668369984 EVENT_LOG_v1 {"time_micros": 1732366930242869, "cf_name": "default", "job": 35738, "event": "table_file_creation", "file_number": 280528, "file_size": 30636586, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4229711594, "largest_seqno": 4259662013, "table_properties": {"data_size": 25928928, "index_size": 1445184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 21269696, "raw_average_key_size": 28, "raw_value_size": 88245004, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 759632, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732366927, "oldest_key_time": 0, "file_creation_time": 1732366927, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280528, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:02:10.268395 139957668369984 [compaction_job.cc:703] job-35738: subcompact[0], size: 0.790982 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35738/att-00 [280508,280509,280510,280511,280512,280513,280514,280515,280516,280517,280518,280519,280520,280521,280522,280523,280524,280525,280526,280527,280528] 2024/11/23-21:02:10.427575 139957668369984 (Original Log Time 2024/11/23-21:00:58.414485) EVENT_LOG_v1 {"time_micros": 1732366858414419, "job": 35738, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [280499], "files_L3": [280504, 280505, 280412, 280413, 280414, 280415, 280416, 280417, 280418, 280419, 280420, 280421, 280422, 280492, 280493, 280494, 280495, 280496, 280497, 280498, 280394, 280395], "score": 0.992987, "input_data_size": 811583488, "oldest_snapshot_seqno": 4259898503} 2024/11/23-21:02:10.427579 139957668369984 (Original Log Time 2024/11/23-21:02:10.425676) [compaction_job.cc:2022] [default] [JOB 35738] Compacted 1@2 + 22@3 files to L3 => 790981707 bytes 2024/11/23-21:02:10.427580 139957668369984 (Original Log Time 2024/11/23-21:02:10.427505) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 21 0 0 0] max score 0.03, MB/sec: 11.3 rd, 11.0 wr, level 3, files in(1, 22) out(21 +0 blob) MB in(23.1, 750.9 +0.0 blob) out(754.3 +0.0 blob), read-write-amplify(66.2) write-amplify(32.7) OK, records in: 19400644, records dropped: 50813 output_compression: Snappy 2024/11/23-21:02:10.427583 139957668369984 (Original Log Time 2024/11/23-21:02:10.427550) EVENT_LOG_v1 {"time_micros": 1732366930427528, "cf": "default", "job": 35738, "event": "compaction_finished", "compaction_time_micros": 71853919, "compaction_time_cpu_micros": 30082853, "output_level": 3, "num_output_files": 21, "total_output_size": 790981707, "num_input_records": 19400644, "num_output_records": 19349831, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 554784143, "file_range_sync_nanos": 0, "file_fsync_nanos": 15636, "file_prepare_write_nanos": 60731, "lsm_state": [0, 0, 0, 21, 0, 0, 0]} 2024/11/23-21:02:10.541497 139957668369984 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732366858,"executesMs":72159,"compactionJobId":35738,"attempt":0,"compactionInputRawBytes":2801086432,"compactionInputZipBytes":811583488,"compactionOutputRawBytes":0,"compactionOutputZipBytes":790981707,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-8b64da6176e29140f8e1c68ebe23d8a1-269895a46ad6de0c-00"}}, response = 2024/11/23-21:02:10.541590 139957668369984 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-35738/att-00: olev 3, work 72.160 s, result 7.834 ms, install 1.923 ms, input{raw 2.609 GiB zip 773.986 MiB}