2024/11/24-04:50:55.074882 139957659977280 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00: fork to child time = 0.007762 sec 2024/11/24-04:50:55.075761 139957659977280 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00 2024/11/24-04:50:55.081403 139957659977280 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00 2024/11/24-04:50:55.082327 139957659977280 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-04:50:55.082576 139957659977280 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-04:50:55.137317 139957659977280 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41803/att-00/MANIFEST-206347 2024/11/24-04:50:55.164537 139957659977280 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41803/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 292581, last_sequence is 4447232924, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 292405 2024/11/24-04:50:55.164544 139957659977280 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-04:50:55.164673 139957659977280 [version_set.cc:5721] Creating manifest 292581 2024/11/24-04:50:55.196244 139957659977280 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-04:50:55.219899 139957659977280 [compaction_job.cc:2437] [default] [JOB 41803] Compacting 1@2 + 16@3 files to L3, score 0.59, subcompactions 1 : 1 2024/11/24-04:50:55.219913 139957659977280 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 31333 Base level 2, inputs: [292548(27MB)], [292456(36MB) 292457(33MB) 292458(34MB) 292459(35MB) 292460(33MB) 292461(35MB) 292462(35MB) 292463(33MB) 292464(35MB) 292465(35MB) 292466(33MB) 292467(35MB) 292468(35MB) 292469(33MB) 292470(35MB) 292571(35MB)] 2024/11/24-04:50:55.226478 139957659977280 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-04:50:55.226508 139957659977280 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-04:50:55.226528 139957659977280 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-04:50:55.226533 139957659977280 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-04:50:55.226545 139957659977280 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-04:50:55.791956 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292582.sst) /dev/shm/ToplingTemp/Topling-273845-W0aIT3 2024/11/24-04:50:56.336930 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292582.sst): first pass time = 0.54's, 246.310'MB/sec 2024/11/24-04:50:57.869850 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292582.sst): old prealloc_size = 147639500, real_size = 33914688 2024/11/24-04:50:57.870445 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292582.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928401 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0319 GB avg-val = 116.57 avg-zval = 34.37 2024/11/24-04:50:57.871262 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292582.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:50:58.721838 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292582.sst): second pass time = 1.53's, 70.640'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.623'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3064.357'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.13's, 301.014'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30723.030'MB/sec zip my value time = 1.53's, unzip length = 0.108'GB zip my value throughput = 70.640'MB/sec zip pipeline throughput = 79.301'MB/sec entries = 928401 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.57 avg-zval = 34.37 usrkeys = 928401 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.57 avg-zval = 34.37 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928401 bytes = 3249424 } seq expand size = 6498807 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.4456 value = 3.3920 dict = 1.00 all = 3.5304 } Zip/UnZip{ index = 0.0957 value = 0.2948 dict = 1.00 all = 0.2833 } ---------------------------- total value len = 2251.440295 GB avg = 0.200 KB (by entry num) total key len = 306.908958 GB avg = 0.027 KB total ukey len = 237.373295 GB avg = 0.015 KB total ukey num = 15.739589564 Billion total entry num = 11.280903512 Billion write speed all = 35.658171424 MB/sec (with seq num) write speed all = 34.400309103 MB/sec (without seq num) 2024/11/24-04:50:58.770004 139957659977280 [compaction_job.cc:1948] [default] [JOB 41803] Generated table #292582: 928401 keys, 38017446 bytes, temperature: kWarm 2024/11/24-04:50:58.770104 139957659977280 EVENT_LOG_v1 {"time_micros": 1732395058770044, "cf_name": "default", "job": 41803, "event": "table_file_creation", "file_number": 292582, "file_size": 38017446, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178518, "largest_seqno": 4447040420, "table_properties": {"data_size": 31904816, "index_size": 1777600, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25995228, "raw_average_key_size": 28, "raw_value_size": 108222530, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928401, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395055, "oldest_key_time": 0, "file_creation_time": 1732395055, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292582, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:50:58.807413 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292583.sst) /dev/shm/ToplingTemp/Topling-273845-0Ibwc3 2024/11/24-04:50:59.754458 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292583.sst): first pass time = 0.95's, 141.728'MB/sec 2024/11/24-04:51:01.207183 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292583.sst): old prealloc_size = 147639500, real_size = 30989904 2024/11/24-04:51:01.207717 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292583.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922824 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0290 GB avg-val = 117.44 avg-zval = 31.42 2024/11/24-04:51:01.208484 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292583.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:51:02.015653 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292583.sst): second pass time = 1.45's, 74.644'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.244'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3449.070'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.339'MB/sec write SST data time = 0.13's, 274.359'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33019.026'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.644'MB/sec zip pipeline throughput = 74.644'MB/sec entries = 922824 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.44 avg-zval = 31.42 usrkeys = 922824 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.44 avg-zval = 31.42 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922824 bytes = 3114544 } seq expand size = 6459768 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0290 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.4460 value = 3.7382 dict = 1.00 all = 3.8391 } Zip/UnZip{ index = 0.0957 value = 0.2675 dict = 1.00 all = 0.2605 } ---------------------------- total value len = 2251.555133 GB avg = 0.200 KB (by entry num) total key len = 306.934797 GB avg = 0.027 KB total ukey len = 237.391752 GB avg = 0.015 KB total ukey num = 15.740512388 Billion total entry num = 11.281826336 Billion write speed all = 35.658495136 MB/sec (with seq num) write speed all = 34.400587665 MB/sec (without seq num) 2024/11/24-04:51:02.060953 139957659977280 [compaction_job.cc:1948] [default] [JOB 41803] Generated table #292583: 922824 keys, 34960670 bytes, temperature: kWarm 2024/11/24-04:51:02.061035 139957659977280 EVENT_LOG_v1 {"time_micros": 1732395062061002, "cf_name": "default", "job": 41803, "event": "table_file_creation", "file_number": 292583, "file_size": 34960670, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4316213719, "largest_seqno": 4447040431, "table_properties": {"data_size": 28992192, "index_size": 1766848, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25839072, "raw_average_key_size": 28, "raw_value_size": 108378752, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922824, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395058, "oldest_key_time": 0, "file_creation_time": 1732395058, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292583, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:51:02.101376 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292584.sst) /dev/shm/ToplingTemp/Topling-273845-grZhV6 2024/11/24-04:51:02.999287 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292584.sst): first pass time = 0.90's, 149.487'MB/sec 2024/11/24-04:51:04.463901 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292584.sst): old prealloc_size = 147639500, real_size = 32809472 2024/11/24-04:51:04.464484 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292584.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923833 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0308 GB avg-val = 117.28 avg-zval = 33.36 2024/11/24-04:51:04.465256 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292584.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:51:05.304071 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292584.sst): second pass time = 1.46's, 74.022'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.921'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3177.290'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.324'MB/sec write SST data time = 0.13's, 292.007'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 33967.229'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 74.022'MB/sec zip pipeline throughput = 74.022'MB/sec entries = 923833 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.28 avg-zval = 33.36 usrkeys = 923833 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.28 avg-zval = 33.36 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923833 bytes = 3233424 } seq expand size = 6466831 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5119 value = 3.5155 dict = 1.00 all = 3.6373 } Zip/UnZip{ index = 0.0951 value = 0.2845 dict = 1.00 all = 0.2749 } ---------------------------- total value len = 2251.669950 GB avg = 0.200 KB (by entry num) total key len = 306.960664 GB avg = 0.027 KB total ukey len = 237.410228 GB avg = 0.015 KB total ukey num = 15.741436221 Billion total entry num = 11.282750169 Billion write speed all = 35.658821598 MB/sec (with seq num) write speed all = 34.400868775 MB/sec (without seq num) 2024/11/24-04:51:05.350554 139957659977280 [compaction_job.cc:1948] [default] [JOB 41803] Generated table #292584: 923833 keys, 36899902 bytes, temperature: kWarm 2024/11/24-04:51:05.350628 139957659977280 EVENT_LOG_v1 {"time_micros": 1732395065350591, "cf_name": "default", "job": 41803, "event": "table_file_creation", "file_number": 292584, "file_size": 36899902, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178530, "largest_seqno": 4447040761, "table_properties": {"data_size": 30820656, "index_size": 1757696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25867324, "raw_average_key_size": 28, "raw_value_size": 108350510, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923833, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395062, "oldest_key_time": 0, "file_creation_time": 1732395062, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292584, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:51:05.388375 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292585.sst) /dev/shm/ToplingTemp/Topling-273845-Ee47V6 2024/11/24-04:51:06.432687 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292585.sst): first pass time = 1.04's, 128.527'MB/sec 2024/11/24-04:51:07.930505 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292585.sst): old prealloc_size = 147639500, real_size = 33040080 2024/11/24-04:51:07.931051 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292585.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927323 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 116.74 avg-zval = 33.46 2024/11/24-04:51:07.931827 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292585.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:51:08.786064 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292585.sst): second pass time = 1.50's, 72.313'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.412'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3380.908'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 292.075'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31229.751'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 72.313'MB/sec zip pipeline throughput = 72.313'MB/sec entries = 927323 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.74 avg-zval = 33.46 usrkeys = 927323 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.74 avg-zval = 33.46 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927323 bytes = 3245648 } seq expand size = 6491261 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.4443 value = 3.4884 dict = 1.00 all = 3.6138 } Zip/UnZip{ index = 0.0957 value = 0.2867 dict = 1.00 all = 0.2767 } ---------------------------- total value len = 2251.784694 GB avg = 0.200 KB (by entry num) total key len = 306.986629 GB avg = 0.027 KB total ukey len = 237.428775 GB avg = 0.015 KB total ukey num = 15.742363544 Billion total entry num = 11.283677492 Billion write speed all = 35.659052171 MB/sec (with seq num) write speed all = 34.401057005 MB/sec (without seq num) 2024/11/24-04:51:08.831602 139957659977280 [compaction_job.cc:1948] [default] [JOB 41803] Generated table #292585: 927323 keys, 37140026 bytes, temperature: kWarm 2024/11/24-04:51:08.831668 139957659977280 EVENT_LOG_v1 {"time_micros": 1732395068831633, "cf_name": "default", "job": 41803, "event": "table_file_creation", "file_number": 292585, "file_size": 37140026, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178548, "largest_seqno": 4447040451, "table_properties": {"data_size": 31032320, "index_size": 1775744, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25965044, "raw_average_key_size": 28, "raw_value_size": 108252690, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927323, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395065, "oldest_key_time": 0, "file_creation_time": 1732395065, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292585, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:51:08.872792 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292586.sst) /dev/shm/ToplingTemp/Topling-273845-pPNpS3 2024/11/24-04:51:09.857981 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292586.sst): first pass time = 0.99's, 136.241'MB/sec 2024/11/24-04:51:11.309333 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292586.sst): old prealloc_size = 147639500, real_size = 31239040 2024/11/24-04:51:11.309840 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292586.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921251 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0292 GB avg-val = 117.69 avg-zval = 31.74 2024/11/24-04:51:11.310598 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292586.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:51:12.120820 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292586.sst): second pass time = 1.45's, 74.745'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.528'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3643.868'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.337'MB/sec write SST data time = 0.13's, 275.565'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 34273.354'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.745'MB/sec zip pipeline throughput = 74.745'MB/sec entries = 921251 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.69 avg-zval = 31.74 usrkeys = 921251 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.69 avg-zval = 31.74 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921251 bytes = 3224400 } seq expand size = 6448757 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4460 value = 3.7074 dict = 1.00 all = 3.8000 } Zip/UnZip{ index = 0.0957 value = 0.2697 dict = 1.00 all = 0.2632 } ---------------------------- total value len = 2251.899566 GB avg = 0.200 KB (by entry num) total key len = 307.012424 GB avg = 0.027 KB total ukey len = 237.447200 GB avg = 0.015 KB total ukey num = 15.743284795 Billion total entry num = 11.284598743 Billion write speed all = 35.659355300 MB/sec (with seq num) write speed all = 34.401315891 MB/sec (without seq num) 2024/11/24-04:51:12.163869 139957659977280 [compaction_job.cc:1948] [default] [JOB 41803] Generated table #292586: 921251 keys, 35320928 bytes, temperature: kWarm 2024/11/24-04:51:12.163932 139957659977280 EVENT_LOG_v1 {"time_micros": 1732395072163896, "cf_name": "default", "job": 41803, "event": "table_file_creation", "file_number": 292586, "file_size": 35320928, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4306988592, "largest_seqno": 4447040461, "table_properties": {"data_size": 29244720, "index_size": 1763840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25795028, "raw_average_key_size": 28, "raw_value_size": 108422798, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921251, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395068, "oldest_key_time": 0, "file_creation_time": 1732395068, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292586, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:51:12.200351 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292587.sst) /dev/shm/ToplingTemp/Topling-273845-DYpYg3 2024/11/24-04:51:13.040949 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292587.sst): first pass time = 0.84's, 159.676'MB/sec 2024/11/24-04:51:14.467837 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292587.sst): old prealloc_size = 147639500, real_size = 33054880 2024/11/24-04:51:14.468524 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292587.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925300 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 117.05 avg-zval = 33.57 2024/11/24-04:51:14.469392 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292587.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:51:15.330049 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292587.sst): second pass time = 1.43's, 75.991'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.087'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2644.349'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.14's, 263.435'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33483.511'MB/sec zip my value time = 1.43's, unzip length = 0.108'GB zip my value throughput = 75.991'MB/sec zip pipeline throughput = 75.991'MB/sec entries = 925300 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.05 avg-zval = 33.57 usrkeys = 925300 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.05 avg-zval = 33.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925300 bytes = 3238560 } seq expand size = 6477100 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5132 value = 3.4868 dict = 1.00 all = 3.6130 } Zip/UnZip{ index = 0.0951 value = 0.2868 dict = 1.00 all = 0.2768 } ---------------------------- total value len = 2252.014352 GB avg = 0.200 KB (by entry num) total key len = 307.038333 GB avg = 0.027 KB total ukey len = 237.465706 GB avg = 0.015 KB total ukey num = 15.744210095 Billion total entry num = 11.285524043 Billion write speed all = 35.659721177 MB/sec (with seq num) write speed all = 34.401634876 MB/sec (without seq num) 2024/11/24-04:51:15.378115 139957659977280 [compaction_job.cc:1948] [default] [JOB 41803] Generated table #292587: 925300 keys, 37148232 bytes, temperature: kWarm 2024/11/24-04:51:15.378184 139957659977280 EVENT_LOG_v1 {"time_micros": 1732395075378149, "cf_name": "default", "job": 41803, "event": "table_file_creation", "file_number": 292587, "file_size": 37148232, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178559, "largest_seqno": 4447038936, "table_properties": {"data_size": 31063120, "index_size": 1760256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25908400, "raw_average_key_size": 28, "raw_value_size": 108309424, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395072, "oldest_key_time": 0, "file_creation_time": 1732395072, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292587, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:51:15.415100 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292588.sst) /dev/shm/ToplingTemp/Topling-273845-69ezl6 2024/11/24-04:51:16.408072 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292588.sst): first pass time = 0.99's, 135.173'MB/sec 2024/11/24-04:51:17.860474 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292588.sst): old prealloc_size = 147639500, real_size = 32733648 2024/11/24-04:51:17.861001 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292588.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927331 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0307 GB avg-val = 116.74 avg-zval = 33.15 2024/11/24-04:51:17.861778 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292588.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:51:18.706727 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292588.sst): second pass time = 1.45's, 74.576'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.527'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3500.301'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 292.184'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34373.087'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.576'MB/sec zip pipeline throughput = 74.576'MB/sec entries = 927331 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.74 avg-zval = 33.15 usrkeys = 927331 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.74 avg-zval = 33.15 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927331 bytes = 3245680 } seq expand size = 6491317 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5149 value = 3.5218 dict = 1.00 all = 3.6439 } Zip/UnZip{ index = 0.0951 value = 0.2839 dict = 1.00 all = 0.2744 } ---------------------------- total value len = 2252.129096 GB avg = 0.200 KB (by entry num) total key len = 307.064298 GB avg = 0.027 KB total ukey len = 237.484253 GB avg = 0.015 KB total ukey num = 15.745137426 Billion total entry num = 11.286451374 Billion write speed all = 35.660003996 MB/sec (with seq num) write speed all = 34.401873518 MB/sec (without seq num) 2024/11/24-04:51:18.757019 139957659977280 [compaction_job.cc:1948] [default] [JOB 41803] Generated table #292588: 927331 keys, 36833814 bytes, temperature: kWarm 2024/11/24-04:51:18.757090 139957659977280 EVENT_LOG_v1 {"time_micros": 1732395078757053, "cf_name": "default", "job": 41803, "event": "table_file_creation", "file_number": 292588, "file_size": 36833814, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178575, "largest_seqno": 4447040477, "table_properties": {"data_size": 30737792, "index_size": 1763840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25965268, "raw_average_key_size": 28, "raw_value_size": 108252510, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927331, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395075, "oldest_key_time": 0, "file_creation_time": 1732395075, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292588, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:51:18.794335 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292589.sst) /dev/shm/ToplingTemp/Topling-273845-dXJdn6 2024/11/24-04:51:19.690298 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292589.sst): first pass time = 0.90's, 149.808'MB/sec 2024/11/24-04:51:21.188485 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292589.sst): old prealloc_size = 147639500, real_size = 31506976 2024/11/24-04:51:21.189040 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292589.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919808 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 117.92 avg-zval = 32.09 2024/11/24-04:51:21.189824 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292589.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:51:22.007639 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292589.sst): second pass time = 1.50's, 72.455'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.443'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3376.214'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.12's, 284.730'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 34398.085'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 72.455'MB/sec zip pipeline throughput = 72.963'MB/sec entries = 919808 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.92 avg-zval = 32.09 usrkeys = 919808 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.92 avg-zval = 32.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919808 bytes = 3219344 } seq expand size = 6438656 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.08 MB all = 0.0356 GB } UnZip/Zip{ index = 10.4444 value = 3.6748 dict = 1.00 all = 3.7717 } Zip/UnZip{ index = 0.0957 value = 0.2721 dict = 1.00 all = 0.2651 } ---------------------------- total value len = 2252.243998 GB avg = 0.200 KB (by entry num) total key len = 307.090052 GB avg = 0.027 KB total ukey len = 237.502649 GB avg = 0.015 KB total ukey num = 15.746057234 Billion total entry num = 11.287371182 Billion write speed all = 35.660323712 MB/sec (with seq num) write speed all = 34.402148570 MB/sec (without seq num) 2024/11/24-04:51:22.053961 139957659977280 [compaction_job.cc:1948] [default] [JOB 41803] Generated table #292589: 919808 keys, 35585082 bytes, temperature: kWarm 2024/11/24-04:51:22.054038 139957659977280 EVENT_LOG_v1 {"time_micros": 1732395082054003, "cf_name": "default", "job": 41803, "event": "table_file_creation", "file_number": 292589, "file_size": 35585082, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4298203002, "largest_seqno": 4447040497, "table_properties": {"data_size": 29515472, "index_size": 1761344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25754624, "raw_average_key_size": 28, "raw_value_size": 108463140, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 919808, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395078, "oldest_key_time": 0, "file_creation_time": 1732395078, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292589, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:51:22.097666 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292590.sst) /dev/shm/ToplingTemp/Topling-273845-iXxF72 2024/11/24-04:51:23.025792 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292590.sst): first pass time = 0.93's, 144.618'MB/sec 2024/11/24-04:51:24.520607 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292590.sst): old prealloc_size = 147639500, real_size = 33110272 2024/11/24-04:51:24.521146 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292590.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926703 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 116.83 avg-zval = 33.58 2024/11/24-04:51:24.522598 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292590.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:51:25.372199 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292590.sst): second pass time = 1.49's, 72.498'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.694'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3411.223'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 295.827'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 26413.371'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.498'MB/sec zip pipeline throughput = 81.352'MB/sec entries = 926703 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.83 avg-zval = 33.58 usrkeys = 926703 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.83 avg-zval = 33.58 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926703 bytes = 3243472 } seq expand size = 6486921 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5131 value = 3.4796 dict = 1.00 all = 3.6072 } Zip/UnZip{ index = 0.0951 value = 0.2874 dict = 1.00 all = 0.2772 } ---------------------------- total value len = 2252.358755 GB avg = 0.200 KB (by entry num) total key len = 307.116000 GB avg = 0.027 KB total ukey len = 237.521183 GB avg = 0.015 KB total ukey num = 15.746983937 Billion total entry num = 11.288297885 Billion write speed all = 35.660612444 MB/sec (with seq num) write speed all = 34.402392991 MB/sec (without seq num) 2024/11/24-04:51:25.418676 139957659977280 [compaction_job.cc:1948] [default] [JOB 41803] Generated table #292590: 926703 keys, 37208010 bytes, temperature: kWarm 2024/11/24-04:51:25.418746 139957659977280 EVENT_LOG_v1 {"time_micros": 1732395085418710, "cf_name": "default", "job": 41803, "event": "table_file_creation", "file_number": 292590, "file_size": 37208010, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178589, "largest_seqno": 4447040880, "table_properties": {"data_size": 31115440, "index_size": 1762944, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25947684, "raw_average_key_size": 28, "raw_value_size": 108270046, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926703, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395082, "oldest_key_time": 0, "file_creation_time": 1732395082, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292590, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:51:25.458660 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292591.sst) /dev/shm/ToplingTemp/Topling-273845-DV0k92 2024/11/24-04:51:26.403648 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292591.sst): first pass time = 0.94's, 142.037'MB/sec 2024/11/24-04:51:27.879936 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292591.sst): old prealloc_size = 147639500, real_size = 32304368 2024/11/24-04:51:27.880502 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292591.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927296 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0303 GB avg-val = 116.74 avg-zval = 32.68 2024/11/24-04:51:27.882559 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292591.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:51:28.733887 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292591.sst): second pass time = 1.48's, 73.386'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 42.632'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3337.616'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 279.122'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34656.424'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.386'MB/sec zip pipeline throughput = 82.622'MB/sec entries = 927296 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.74 avg-zval = 32.68 usrkeys = 927296 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.74 avg-zval = 32.68 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927296 bytes = 3245552 } seq expand size = 6491072 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5122 value = 3.5718 dict = 1.00 all = 3.6869 } Zip/UnZip{ index = 0.0951 value = 0.2800 dict = 1.00 all = 0.2712 } ---------------------------- total value len = 2252.473500 GB avg = 0.200 KB (by entry num) total key len = 307.141964 GB avg = 0.027 KB total ukey len = 237.539729 GB avg = 0.015 KB total ukey num = 15.747911233 Billion total entry num = 11.289225181 Billion write speed all = 35.660902639 MB/sec (with seq num) write speed all = 34.402638761 MB/sec (without seq num) 2024/11/24-04:51:28.781116 139957659977280 [compaction_job.cc:1948] [default] [JOB 41803] Generated table #292591: 927296 keys, 36404438 bytes, temperature: kWarm 2024/11/24-04:51:28.781182 139957659977280 EVENT_LOG_v1 {"time_micros": 1732395088781149, "cf_name": "default", "job": 41803, "event": "table_file_creation", "file_number": 292591, "file_size": 36404438, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178599, "largest_seqno": 4447040513, "table_properties": {"data_size": 30308128, "index_size": 1764224, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25964288, "raw_average_key_size": 28, "raw_value_size": 108253584, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927296, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395085, "oldest_key_time": 0, "file_creation_time": 1732395085, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292591, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:51:28.816920 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292592.sst) /dev/shm/ToplingTemp/Topling-273845-GLK4P3 2024/11/24-04:51:29.776329 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292592.sst): first pass time = 0.96's, 139.903'MB/sec 2024/11/24-04:51:31.444035 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292592.sst): old prealloc_size = 147639500, real_size = 31496000 2024/11/24-04:51:31.444570 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292592.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.08 2024/11/24-04:51:31.446554 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292592.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:51:32.264510 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292592.sst): second pass time = 1.67's, 65.081'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.358'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3502.253'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.13's, 277.037'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 33081.590'MB/sec zip my value time = 1.67's, unzip length = 0.108'GB zip my value throughput = 65.081'MB/sec zip pipeline throughput = 129.964'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 32.08 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 32.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0356 GB } UnZip/Zip{ index = 10.3772 value = 3.6779 dict = 1.00 all = 3.7731 } Zip/UnZip{ index = 0.0964 value = 0.2719 dict = 1.00 all = 0.2650 } ---------------------------- total value len = 2252.588412 GB avg = 0.200 KB (by entry num) total key len = 307.167705 GB avg = 0.027 KB total ukey len = 237.558115 GB avg = 0.015 KB total ukey num = 15.748830533 Billion total entry num = 11.290144481 Billion write speed all = 35.661108095 MB/sec (with seq num) write speed all = 34.402803649 MB/sec (without seq num) 2024/11/24-04:51:32.313417 139957659977280 [compaction_job.cc:1948] [default] [JOB 41803] Generated table #292592: 919300 keys, 35572722 bytes, temperature: kWarm 2024/11/24-04:51:32.313483 139957659977280 EVENT_LOG_v1 {"time_micros": 1732395092313448, "cf_name": "default", "job": 41803, "event": "table_file_creation", "file_number": 292592, "file_size": 35572722, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4290156447, "largest_seqno": 4442090609, "table_properties": {"data_size": 29494192, "index_size": 1771776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395088, "oldest_key_time": 0, "file_creation_time": 1732395088, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292592, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:51:32.358936 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292593.sst) /dev/shm/ToplingTemp/Topling-273845-Me5vB3 2024/11/24-04:51:33.304258 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292593.sst): first pass time = 0.95's, 141.986'MB/sec 2024/11/24-04:51:34.941443 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292593.sst): old prealloc_size = 147639500, real_size = 33224336 2024/11/24-04:51:34.941956 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292593.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927355 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 116.73 avg-zval = 33.66 2024/11/24-04:51:34.944092 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292593.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:51:35.810255 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292593.sst): second pass time = 1.64's, 66.150'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.967'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3588.243'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.13's, 282.833'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 29284.796'MB/sec zip my value time = 1.64's, unzip length = 0.108'GB zip my value throughput = 66.150'MB/sec zip pipeline throughput = 129.186'MB/sec entries = 927355 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.73 avg-zval = 33.66 usrkeys = 927355 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.73 avg-zval = 33.66 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927355 bytes = 3245760 } seq expand size = 6491485 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4462 value = 3.4677 dict = 1.00 all = 3.5956 } Zip/UnZip{ index = 0.0957 value = 0.2884 dict = 1.00 all = 0.2781 } ---------------------------- total value len = 2252.703156 GB avg = 0.200 KB (by entry num) total key len = 307.193671 GB avg = 0.027 KB total ukey len = 237.576662 GB avg = 0.015 KB total ukey num = 15.749757888 Billion total entry num = 11.291071836 Billion write speed all = 35.661306804 MB/sec (with seq num) write speed all = 34.402961163 MB/sec (without seq num) 2024/11/24-04:51:35.855825 139957659977280 [compaction_job.cc:1948] [default] [JOB 41803] Generated table #292593: 927355 keys, 37328778 bytes, temperature: kWarm 2024/11/24-04:51:35.855892 139957659977280 EVENT_LOG_v1 {"time_micros": 1732395095855859, "cf_name": "default", "job": 41803, "event": "table_file_creation", "file_number": 292593, "file_size": 37328778, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178645, "largest_seqno": 4447040528, "table_properties": {"data_size": 31216832, "index_size": 1775488, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25965940, "raw_average_key_size": 28, "raw_value_size": 108251890, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927355, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395092, "oldest_key_time": 0, "file_creation_time": 1732395092, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292593, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:51:35.892264 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292594.sst) /dev/shm/ToplingTemp/Topling-273845-uALrp3 2024/11/24-04:51:36.823502 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292594.sst): first pass time = 0.93's, 144.134'MB/sec 2024/11/24-04:51:38.375737 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292594.sst): old prealloc_size = 147639500, real_size = 32133296 2024/11/24-04:51:38.376280 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292594.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927291 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0301 GB avg-val = 116.74 avg-zval = 32.50 2024/11/24-04:51:38.378348 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292594.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:51:39.231961 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292594.sst): second pass time = 1.55's, 69.773'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.319'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3394.187'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.14's, 267.397'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34282.840'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 69.773'MB/sec zip pipeline throughput = 105.372'MB/sec entries = 927291 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.74 avg-zval = 32.50 usrkeys = 927291 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.74 avg-zval = 32.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927291 bytes = 3245536 } seq expand size = 6491037 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.5118 value = 3.5921 dict = 1.00 all = 3.7043 } Zip/UnZip{ index = 0.0951 value = 0.2784 dict = 1.00 all = 0.2700 } ---------------------------- total value len = 2252.817900 GB avg = 0.200 KB (by entry num) total key len = 307.219635 GB avg = 0.027 KB total ukey len = 237.595208 GB avg = 0.015 KB total ukey num = 15.750685179 Billion total entry num = 11.291999127 Billion write speed all = 35.661567107 MB/sec (with seq num) write speed all = 34.403178106 MB/sec (without seq num) 2024/11/24-04:51:39.279220 139957659977280 [compaction_job.cc:1948] [default] [JOB 41803] Generated table #292594: 927291 keys, 36233306 bytes, temperature: kWarm 2024/11/24-04:51:39.279296 139957659977280 EVENT_LOG_v1 {"time_micros": 1732395099279261, "cf_name": "default", "job": 41803, "event": "table_file_creation", "file_number": 292594, "file_size": 36233306, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178655, "largest_seqno": 4447039322, "table_properties": {"data_size": 30136992, "index_size": 1764288, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25964148, "raw_average_key_size": 28, "raw_value_size": 108253718, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927291, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395095, "oldest_key_time": 0, "file_creation_time": 1732395095, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292594, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:51:39.315140 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292595.sst) /dev/shm/ToplingTemp/Topling-273845-cPi8B5 2024/11/24-04:51:40.192591 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292595.sst): first pass time = 0.88's, 152.970'MB/sec 2024/11/24-04:51:41.697639 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292595.sst): old prealloc_size = 147639500, real_size = 31420512 2024/11/24-04:51:41.698196 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292595.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 32.01 2024/11/24-04:51:41.700270 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292595.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:51:42.535406 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292595.sst): second pass time = 1.50's, 72.111'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.373'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3282.529'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.13's, 271.994'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 38512.405'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 72.111'MB/sec zip pipeline throughput = 105.595'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.01 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.01 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4458 value = 3.6859 dict = 1.00 all = 3.7811 } Zip/UnZip{ index = 0.0957 value = 0.2713 dict = 1.00 all = 0.2645 } ---------------------------- total value len = 2252.932813 GB avg = 0.199 KB (by entry num) total key len = 307.245375 GB avg = 0.027 KB total ukey len = 237.613594 GB avg = 0.015 KB total ukey num = 15.751604479 Billion total entry num = 11.292918427 Billion write speed all = 35.661885353 MB/sec (with seq num) write speed all = 34.403451814 MB/sec (without seq num) 2024/11/24-04:51:42.580411 139957659977280 [compaction_job.cc:1948] [default] [JOB 41803] Generated table #292595: 919300 keys, 35497202 bytes, temperature: kWarm 2024/11/24-04:51:42.580478 139957659977280 EVENT_LOG_v1 {"time_micros": 1732395102580447, "cf_name": "default", "job": 41803, "event": "table_file_creation", "file_number": 292595, "file_size": 35497202, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4281447121, "largest_seqno": 4433580526, "table_properties": {"data_size": 29430352, "index_size": 1760128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395099, "oldest_key_time": 0, "file_creation_time": 1732395099, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292595, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:51:42.617471 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292596.sst) /dev/shm/ToplingTemp/Topling-273845-iThar6 2024/11/24-04:51:43.496640 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292596.sst): first pass time = 0.88's, 152.671'MB/sec 2024/11/24-04:51:45.048317 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292596.sst): old prealloc_size = 147639500, real_size = 33309296 2024/11/24-04:51:45.048852 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292596.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926942 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.80 avg-zval = 33.77 2024/11/24-04:51:45.050942 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292596.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:51:45.927966 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292596.sst): second pass time = 1.55's, 69.810'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 42.527'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3486.651'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.310'MB/sec write SST data time = 0.13's, 286.083'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32630.332'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 69.810'MB/sec zip pipeline throughput = 90.022'MB/sec entries = 926942 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.80 avg-zval = 33.77 usrkeys = 926942 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.80 avg-zval = 33.77 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926942 bytes = 3244304 } seq expand size = 6488594 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4431 value = 3.4587 dict = 1.00 all = 3.5880 } Zip/UnZip{ index = 0.0958 value = 0.2891 dict = 1.00 all = 0.2787 } ---------------------------- total value len = 2253.047565 GB avg = 0.199 KB (by entry num) total key len = 307.271330 GB avg = 0.027 KB total ukey len = 237.632132 GB avg = 0.015 KB total ukey num = 15.752531421 Billion total entry num = 11.293845369 Billion write speed all = 35.662160048 MB/sec (with seq num) write speed all = 34.403682686 MB/sec (without seq num) 2024/11/24-04:51:45.976983 139957659977280 [compaction_job.cc:1948] [default] [JOB 41803] Generated table #292596: 926942 keys, 37407824 bytes, temperature: kWarm 2024/11/24-04:51:45.977058 139957659977280 EVENT_LOG_v1 {"time_micros": 1732395105977023, "cf_name": "default", "job": 41803, "event": "table_file_creation", "file_number": 292596, "file_size": 37407824, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178675, "largest_seqno": 4447040819, "table_properties": {"data_size": 31302176, "index_size": 1775232, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25954376, "raw_average_key_size": 28, "raw_value_size": 108263404, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926942, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395102, "oldest_key_time": 0, "file_creation_time": 1732395102, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292596, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:51:46.025466 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292597.sst) /dev/shm/ToplingTemp/Topling-273845-3YRlX5 2024/11/24-04:51:46.925627 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292597.sst): first pass time = 0.90's, 149.112'MB/sec 2024/11/24-04:51:48.384675 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292597.sst): old prealloc_size = 147639500, real_size = 31766672 2024/11/24-04:51:48.385253 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292597.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927124 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0298 GB avg-val = 116.77 avg-zval = 32.11 2024/11/24-04:51:48.387612 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292597.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:51:49.229338 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292597.sst): second pass time = 1.46's, 74.237'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.463'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3202.987'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.13's, 272.832'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34077.651'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 74.237'MB/sec zip pipeline throughput = 74.237'MB/sec entries = 927124 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.77 avg-zval = 32.11 usrkeys = 927124 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.77 avg-zval = 32.11 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927124 bytes = 3244944 } seq expand size = 6489868 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0298 GB dict = 1.08 MB all = 0.0359 GB } UnZip/Zip{ index = 10.5126 value = 3.6364 dict = 1.00 all = 3.7422 } Zip/UnZip{ index = 0.0951 value = 0.2750 dict = 1.00 all = 0.2672 } ---------------------------- total value len = 2253.162313 GB avg = 0.199 KB (by entry num) total key len = 307.297289 GB avg = 0.027 KB total ukey len = 237.650675 GB avg = 0.015 KB total ukey num = 15.753458545 Billion total entry num = 11.294772493 Billion write speed all = 35.662480029 MB/sec (with seq num) write speed all = 34.403957229 MB/sec (without seq num) 2024/11/24-04:51:49.276366 139957659977280 [compaction_job.cc:1948] [default] [JOB 41803] Generated table #292597: 927124 keys, 35866146 bytes, temperature: kWarm 2024/11/24-04:51:49.276436 139957659977280 EVENT_LOG_v1 {"time_micros": 1732395109276400, "cf_name": "default", "job": 41803, "event": "table_file_creation", "file_number": 292597, "file_size": 35866146, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178686, "largest_seqno": 4447040854, "table_properties": {"data_size": 29770880, "index_size": 1763840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25959472, "raw_average_key_size": 28, "raw_value_size": 108258384, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927124, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395106, "oldest_key_time": 0, "file_creation_time": 1732395106, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292597, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:51:49.313105 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292598.sst) /dev/shm/ToplingTemp/Topling-273845-G9THo3 2024/11/24-04:51:49.406753 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292598.sst): first pass time = 0.09's, 216.967'MB/sec 2024/11/24-04:51:49.608809 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292598.sst): old prealloc_size = 147639500, real_size = 5996928 2024/11/24-04:51:49.608937 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292598.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 139100 prefix = 4 raw-key = 0.0022 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0164 GB zip-val = 0.0057 GB avg-val = 118.00 avg-zval = 40.94 2024/11/24-04:51:49.609109 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292598.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.020 GB 2024/11/24-04:51:49.733670 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00/292598.sst): second pass time = 0.20's, 81.524'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.12's, 52.632'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2524.702'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.332'MB/sec write SST data time = 0.02's, 327.128'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.172'MB, throughput = 5736.821'MB/sec zip my value time = 0.20's, unzip length = 0.016'GB zip my value throughput = 81.524'MB/sec zip pipeline throughput = 81.524'MB/sec entries = 139100 avg-key = 28.00 avg-zkey = 1.92 avg-val = 118.00 avg-zval = 40.94 usrkeys = 139100 avg-key = 20.00 avg-zkey = 1.92 avg-val = 125.00 avg-zval = 40.94 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 139100 bytes = 434704 } seq expand size = 973700 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0028 GB value = 0.0164 GB dict = 0.17 MB all = 0.0203 GB } __ZipSize{ index = 0.0003 GB value = 0.0057 GB dict = 0.17 MB all = 0.0066 GB } UnZip/Zip{ index = 10.3992 value = 2.8824 dict = 1.00 all = 3.0909 } Zip/UnZip{ index = 0.0962 value = 0.3469 dict = 1.00 all = 0.3235 } ---------------------------- total value len = 2253.179701 GB avg = 0.199 KB (by entry num) total key len = 307.301184 GB avg = 0.027 KB total ukey len = 237.653457 GB avg = 0.015 KB total ukey num = 15.753597645 Billion total entry num = 11.294911593 Billion write speed all = 35.662525941 MB/sec (with seq num) write speed all = 34.403996482 MB/sec (without seq num) 2024/11/24-04:51:49.748850 139957659977280 [compaction_job.cc:1948] [default] [JOB 41803] Generated table #292598: 139100 keys, 6570529 bytes, temperature: kWarm 2024/11/24-04:51:49.748914 139957659977280 EVENT_LOG_v1 {"time_micros": 1732395109748882, "cf_name": "default", "job": 41803, "event": "table_file_creation", "file_number": 292598, "file_size": 6570529, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4276369594, "largest_seqno": 4299726142, "table_properties": {"data_size": 5694448, "index_size": 267520, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3894800, "raw_average_key_size": 28, "raw_value_size": 16413800, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 139100, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395109, "oldest_key_time": 0, "file_creation_time": 1732395109, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292598, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:51:49.761568 139957659977280 [compaction_job.cc:703] job-41803: subcompact[0], size: 0.589995 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41803/att-00 [292582,292583,292584,292585,292586,292587,292588,292589,292590,292591,292592,292593,292594,292595,292596,292597,292598] 2024/11/24-04:51:49.862895 139957659977280 (Original Log Time 2024/11/24-04:50:55.219965) EVENT_LOG_v1 {"time_micros": 1732395055219925, "job": 41803, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [292548], "files_L3": [292456, 292457, 292458, 292459, 292460, 292461, 292462, 292463, 292464, 292465, 292466, 292467, 292468, 292469, 292470, 292571], "score": 0.59455, "input_data_size": 612945272, "oldest_snapshot_seqno": 4447232851} 2024/11/24-04:51:49.862899 139957659977280 (Original Log Time 2024/11/24-04:51:49.861476) [compaction_job.cc:2022] [default] [JOB 41803] Compacted 1@2 + 16@3 files to L3 => 589995055 bytes 2024/11/24-04:51:49.862901 139957659977280 (Original Log Time 2024/11/24-04:51:49.862829) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 17 0 0 0] max score 0.02, MB/sec: 11.2 rd, 10.8 wr, level 3, files in(1, 16) out(17 +0 blob) MB in(27.1, 557.4 +0.0 blob) out(562.7 +0.0 blob), read-write-amplify(42.3) write-amplify(20.8) OK, records in: 15029423, records dropped: 92941 output_compression: Snappy 2024/11/24-04:51:49.862906 139957659977280 (Original Log Time 2024/11/24-04:51:49.862874) EVENT_LOG_v1 {"time_micros": 1732395109862853, "cf": "default", "job": 41803, "event": "compaction_finished", "compaction_time_micros": 54541612, "compaction_time_cpu_micros": 22570263, "output_level": 3, "num_output_files": 17, "total_output_size": 589995055, "num_input_records": 15029423, "num_output_records": 14936482, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 409165806, "file_range_sync_nanos": 0, "file_fsync_nanos": 11277, "file_prepare_write_nanos": 55478, "lsm_state": [0, 0, 0, 17, 0, 0, 0]} 2024/11/24-04:51:50.001302 139957659977280 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732395055,"executesMs":54778,"compactionJobId":41803,"attempt":0,"compactionInputRawBytes":2180991098,"compactionInputZipBytes":612945272,"compactionOutputRawBytes":0,"compactionOutputZipBytes":589995055,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-7fd6d1da84c33e359b1d61727cf9eeac-71cbd4cc2ba5a32e-00"}}, response = 2024/11/24-04:51:50.001371 139957659977280 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41803/att-00: olev 3, work 54.779 s, result 7.737 ms, install 1.447 ms, input{raw 2.031 GiB zip 584.550 MiB}